proposal: testing/nettest: in-memory implementations of net package interfaces

[neild]

This is a proposal to add in-memory implementations of net.Listener, net.Conn, and net.PacketConn to the standard library.

Motivation

The net package defines abstract interfaces describing stream and packet-oriented connections: Listener, Conn, and PacketConn. It defines concrete types implementing these interfaces for TCP, Unix, and UDP sockets.

While it is often useful to test network programs using real, loopback network connections, it is also often useful to use a fake network implementation.

• The testing/synctest package does not work well with real network connections.
• Tests using real connections are often subject to port exhaustion and other sources of flakiness.
• In-memory fakes can be significantly faster than real connections.
• Fake connections allow injecting errors and other behaviors.

The net package does provide an in-memory implementation of one network interface: net.Pipe creates an in-memory net.Conn. Unlike a TCP connection, the connection created by net.Pipe is synchronous with writes on one end blocking until they are matched with a write on the other and vice-versa. This property can make net.Pipe tricky to use in tests, since code which reasonably assumes the presence of some amount of network buffer may deadlock when used with an unbuffered pipe.

Some prior related issues:

• net/http/httptest: optional faster test server #14200 - optionally use in-memory networking for httptest.Server for speed and to avoid ephemeral port exhaustion
• proposal: net/v2: make Pipe asynchronous #24205 - make net.Pipe asynchronous to better emulate real TCP connections
• proposal: net/http/httptest: synctest support #76608 - make httptest.Server work with synctest

Design goals

The goal of this proposal is to provide robust, general-purpose implementations of Listener, Conn, and PacketConn, suitable for use in most test environments. The net/http and golang.org/x/net/http2 packages contain internal implementations of Listener and Conn, and the following design is informed by the features we have found useful in testing those packages.

It is not a goal to provide implementations that will suit all conceivable needs. Tests which require advanced precise control over the behavior of connections (for example, simulating latency or packet loss) may still need to use their own test fixtures that provide that control.

In particular, the following proposal provides:

• Control over the simulated IP addresses used by connections.
• Control over the size of network buffers. Some amount of buffering is necessary to emulate the behavior of real-world networking stacks. Since there isn't a single, obviously correct buffer size, we will choose a reasonable default and let the user override it.
• The ability to inject errors. Many tests exercise error handling paths.

It does not provide:

• Emulation of the errors returned by specific platforms, such as syscall.ECONNREFUSED. Different platforms have different behavior, perfect emulation of corner cases is very difficult, and rather than providing an imperfect emulation of a single platform's behavior we will not attempt to emulate any of them.
• Simulated latency or packet loss.

Proposal

We add a new package, testing/nettest.

The nettest package contains concrete Listener, Conn, and PacketConn types which implement the net package interfaces of the same names:

package nettest

type Listener struct { /* ... */ }

func (*Listener) Accept() (net.Conn, error) // always a *Conn
func (*Listener) Close() error
func (*Listener) Addr() net.Addr

type Conn struct { /* ... */ }

func (*Conn) Read(b []byte) (n int, err error)
func (*Conn) Write(b []byte) (n int, err error)
func (*Conn) Close() error
func (*Conn) LocalAddr() net.Addr
func (*Conn) RemoteAddr() net.Addr
func (*Conn) SetDeadline(t time.Time) error
func (*Conn) SetReadDeadline(t time.Time) error
func (*Conn) SetWriteDeadline(t time.Time) error

type PacketConn struct { /* ... */ }

func (*PacketConn) ReadFrom(p []byte) (n int, addr net.Addr, err error)
func (*PacketConn) WriteTo(p []byte, addr net.Addr) (n int, err error)
func (*PacketConn) Close() error
func (*PacketConn) LocalAddr() net.Addr
func (*PacketConn) SetDeadline(t time.Time) error
func (*PacketConn) SetReadDeadline(t time.Time) error
func (*PacketConn) SetWriteDeadline(t time.Time) error

In addition, Conn supports the CloseWrite and CloseRead methods from *net.TCPConn:

func (*Conn) CloseRead() error
func (*Conn) CloseWrite() error

Connections naturally come in pairs, where each side reads data written by the other. The Conn.Peer method returns the other end of the connection. This isn't strictly necessary, but we have found it very convenient in some net/http tests.

// Peer returns the other side of the connection.
func (*Conn) Peer() *Conn

Creating connections

The NewConnPair function creates new connections.

// NewConnPair returns a pair of connected Conns.
func NewConnPair() (*Conn, *Conn)

The NewListener function creates a new listener, and Listener.NewConn creates a new connection to the listener.

In some cases, a test may need to customize a connection before it is returned by Listener.Accept. For example, a test may want to create a connection from a specific address. Listener.NewConnConfig provides the ability to modify a connection before it is accepted by the listener.

// NewListener returns a new Listener.
func NewListener() *Listener

// NewConn returns a new connection to the listener.
//
// Accept will return the other side of the conn.
func (*Listener) NewConn() *Conn

// NewConnConfig returns a new connection to the listener.
//
// The function f is called with the new client connection.
// After f returns, Accept will return the other side of the conn.
//
// For example, to create a connection from a specific IP address:
//
//	conn := listener.NewConnConfig(func(conn *nettest.Conn) {
//		conn.SetLocalAddr(someAddress)
//	}
func (*Listener) NewConnConfig(f func(*nettest.Conn)) *Conn

Packet connections are more complicated, because a PacketConn is not connected to a single peer. The PacketNet type contains a group of communicating packet connections.

// A PacketNet is a group of communicating [PacketConn]s.
type PacketNet struct{}

// NewPacketNet returns a new PacketNet.
func NewPacketNet() *PacketNet

// NewConn returns a new [PacketConn] listening on the given address.
// It returns an error if there is an existing listener on this address.
func (*PacketNet) NewConn(a netip.AddrPort) (*PacketConn, error)

Addresses

Methods which return a net.Addr (an interface type) will return a *net.TCPAddr or *net.UDPAddr as appropriate.

Listener and Conn have methods which permit the user to set these addresses. Since these methods don't need to match existing net package interfaces, they take a netip.AddrPort. There is no need for a PacketConn.SetLocalAddr, since the address is always set at creation time.

// SetAddr sets the address returned by Addr.
func (*Listener) SetAddr(a netip.AddrPort)

// SetLocalAddr sets the address returned by LocalAddr.
//
// To set the address returned by RemoteAddr, use conn.Peer().SetLocalAddr(addr).
func (*Conn) SetLocalAddr(a netip.AddrPort)

Errors

Tests often need to inject faults into their fakes. Listeners and connections will support setting an error to be returned by various methods.

Injected errors are wrapped in a *net.OpError as appropriate.

Closing a listener or connection overrides injected errors. For example, Conn.Read will always return net.ErrClosed after Conn.Close is called.

// SetAcceptError causes any currently blocked and future Accept calls to return
// a net.OpError wrapping err.
//
// If the listener's accept queue contains a connection,
// Accept will return it before before the accept error.
//
// A nil error restores the usual behavior.
func (*Listener) SetAcceptError(error)

// SetCloseError sets the error returned by Close.
// A nil error restores the usual behavior.
func (*Listener) SetCloseError(error)

// SetReadError causes any currently blocked and future Read calls to return
// a net.OpError wrapping err. It does not affect the other side of the connection.
// A nil error restores the usual behavior.
func (*Conn) SetReadError(error)

// SetWriteError causes any currently blocked and future Write calls to return
// a net.OpError wrapping err. It does not affect the other side of the connection.
// A nil error restores the usual behavior.
func (*Conn) SetWriteError(error)

// SetCloseError sets the error returned by Close.
// A nil error restores the usual behavior.
func (*Conn) SetCloseError(error)

// SetReadError causes any currently blocked and future ReadFrom calls to return
// a net.OpError wrapping err.
//
// If the connection's receive buffer contains data,
// Read will return it before the read error.
//
// A nil error restores the usual behavior.
func (*PacketConn) SetReadError(error)

// SetWriteError causes future WriteTo calls to return
// a net.OpError wrapping err.
// A nil error restores the usual behavior.
func (*PacketConn) SetWriteError(error)

// SetCloseError sets the error returned by Close.
// A nil error restores the usual behavior.
func (*PacketConn) SetCloseError(error)

This feature is an unfortunately large amount of API surface (eight methods), but fault injection is a common enough need that it seems worth including.

We do not support setting an error for SetDeadline, SetReadDeadline, and SetWriteDeadline, because while they do return an error in practice these methods never fail.

We do not support setting an error for CloseRead and CloseWrite because the real version of these functions (implemented using shutdown(2)) can only return an error when the underlying TCP socket has been closed. In contrast, Conn.Close can return an error when a prior buffered write cannot be flushed.

Buffers

TCP connections are fundamentally buffered. Simulating real connection behavior requires that nettest.Conn be buffered. The SetReadBufferSize method gives the user control over that buffer size.

// SetReadBufferSize sets the connection's read buffer size.
// Writes to the other side of the connection will block when the buffer is full.
// A size of 0 blocks all writes until the size is increased.
// A negative value makes the buffer unlimited (the default).
func (*Conn) SetReadBufferSize(size int)

We use this feature in net/http tests to test various scenarios where writes block.

We will not provide control over the size of a Listener's accept queue or a PacketConn's receive buffer. Exceeding a TCP listener's queue results in a connection error, but nettest does not contain a good place to return this error. Exceeding a UDP receiver's buffer results in dropped packets, and simulated packet loss is out of scope for nettest (at least in its initial version).

Introspection

Tests sometimes want to ask questions about the behavior of the system under test that are clumsy to answer with the standard net.Conn interface: Has the peer closed the connection or not? Has the peer finished writing to the connection or not? The testing/synctest package makes these questions answerable, but it is still clumsy to assert a negative because Conn.Read is a blocking operation.

Listener, Conn, and PacketConn all have methods which provide a non-blocking way to test their readability and close status.

// CanAccept reports whether [Accept] can return a connection or error.
// If [Accept] would block, CanAccept returns false.
func (*Listener) CanAccept() bool

// CanRead reports whether [Read] can return at least one byte or an error.
// If [Read] would block, CanRead returns false.
func (*Conn) CanRead() bool

// CanRead reports whether [ReadFrom] can return at least one byte or an error.
// If [ReadFrom] would block, CanRead returns false.
func (*PacketConn) CanRead() bool

// IsClosed reports whether the listener has been closed.
func (*Listener) IsClosed() bool

// IsClosed reports whether the connection has been closed.
// A connection is closed if [CloseRead] and [CloseWrite] are both called,
// or if [Close] is called.
//
// To identify when the other side of the Conn has been closed,
// use Conn.Peer().IsClosed().
func (*Conn) IsClosed() bool

// IsClosed reports whether the connection has been closed.
func (*PacketConn) IsClosed() bool

There is no Conn.CanWrite because connection writability is not a useful property to examine in tests: It depends on the connection buffer size (which is under control of tests but will vary in real environments) and is unlikely to provide useful information about the system under tests. (This rationale is subtle; perhaps we should have a Conn.CanWrite method just to avoid the need to answer questions about why it doesn't exist.)

Experimentation

The proposal is quite large, which makes it likely that we will discover problems with it only after users have a chance to try it out.

I propose that we initially add this package to the x/exp repository as golang.org/x/exp/testing/nettest. Once we have developed confidence with the design, we will deprecate the x/exp package and add it to the standard library as testing/nettest.

[neild]

https://go.dev/cl/740100 contains a draft implementation of this proposal.

[gopherbot]

Change https://go.dev/cl/740100 mentions this issue: testing/nettest: add in-memory test implementations of common net interfaces

[Merovius]

The net package defines abstract interfaces describing stream and packet-oriented connections: Listener, Conn, and PacketConn. It defines concrete types implementing these interfaces for TCP, Unix, and UDP sockets.

The proposal seems to exclusively support TCP and UDP. Was it a conscious decision to exclude Unix? I understand that it's platform-specific, but it still seems like a fairly common connection type.

[FiloSottile]

Building the platform connection pairs for the crypto/tls and x/crypto/ssh tests is a major pain source, this would be great!

[jaloren]

To reduce the errors api surface what if it was collapsed into a single function

func (*Conn) SetErrors(errConfig)

And the user just sets what errors they want on the struct ?

[neild]

@Merovius What would support for Unix connections look like?

The current scope of this proposal is to provide test implementations of net.Conn, net.Listener, and net.PacketConn. These interfaces include methods that operate on net.Addrs, and the current form of this proposal dictates that those addresses are always *net.TCPAddr or *net.UDPAddr.

We could drop that restriction and leave the type of the address up to the user, changing the following functions which take a netip.AddrPort to take a net.Addr instead:

func (*Listener) SetAddr(a net.Addr)
func (*Conn) SetLocalAddr(a net.Addr)
func (*PacketNet) NewConn(a net.Addr) (*PacketConn, error)

PacketNet needs a comparable address type and net.Addr is not comparable, but we could say that it compares addresses using the tuple of (a.Network(), a.String()).

Would that suffice?

[Merovius]

@neild Yes, seeing the part about *net.{TCP,UDP}Addr was what prompted my question.

There's more useful stuff that Unix connection can do (like sending file descriptors), but that seems overkill. I was just wondering because the constraint to TCP/UDP addresses seemed artificial.