SEANK.H.LIAO

reusing tcp ports

how to reuse ports for TCP in Go

TCP port reuse

Usually, when you make outgoing TCP connections, you don't care about the source address, and you can just get a free ephemeral one.

Sometimes you do, and you want all your connections to reuse the same source ip:port, after all, TCP connections only need to be a unique 4 tuple.

Sometimes you want even more, you want to accept incoming connections on a port, and also make outgoing connections from the same port.

Basics

The long way round, with syscalls

Normal dial
 1func dial(la *syscall.SockaddrInet4) net.Conn {
 2        syscall.ForkLock.Lock()
 3        fd, err := syscall.Socket(syscall.AF_INET, syscall.SOCK_STREAM, 0)
 4        if err != nil {
 5                panic(err)
 6        }
 7        syscall.ForkLock.Unlock()
 8
 9        err = syscall.Bind(fd, la)
10        if err != nil {
11                panic(err)
12        }
13
14        ra := &syscall.SockaddrInet4{Port: 9999}
15        copy(ra.Addr[:], net.IPv4(127, 0, 0, 1))
16
17        err := syscall.Connect(fd, ra)
18        if err != nil {
19                panic(err)
20        }
21
22        conn, err := net.FileConn(os.NewFile(uintptr(fd), ""))
23        if err != nil {
24                panic(err)
25        }
26
27        conn.Write([]byte("foo bar"))
28}
Normal listen
 1func listen(sa *syscall.SockaddrInet4) {
 2        syscall.ForkLock.Lock()
 3        fd, err := syscall.Socket(syscall.AF_INET, syscall.SOCK_STREAM, 0)
 4        if err != nil {
 5                panic(err)
 6        }
 7        syscall.ForkLock.Unlock()
 8
 9        err = syscall.Bind(fd, la)
10        if err != nil {
11                panic(err)
12        }
13
14        err = syscall.Listen(fd, 10)
15        if err != nil {
16                panic(err)
17        }
18
19        l, err := net.FileListener(os.NewFile(uintptr(fd), ""))
20        if err != nil {
21                panic(err)
22        }
23
24        for {
25                conn, err := l.Accept()
26                if err != nil {
27                        panic(err)
28                }
29                conn.Write([]byte("hello world"))
30        }
31}
Options

setsockopt: modify how the socket will behave, use before bind

there are 2 options we care about:

1err = syscall.SetsockoptInt(fd, syscall.SOL_SOCKET, unix.SO_REUSEADDR|unix.SO_REUSEPORT, 1)
2if err != nil {
3  panic(err)
4}

net

Thankfully net is flexible enough that we don't need to do everything with syscalls

dialer
 1func dial() {
 2        la, err := net.ResolveTCPAddr("tcp", "127.0.0.1:9999")
 3        if err != nil {
 4                panic(err)
 5        }
 6
 7        d := &net.Dialer{
 8                LocalAddr: la,
 9                Control: func(network, address string, c syscall.RawConn) error {
10                        var err error
11                        c.Control(func(fd uintptr) {
12                                err = syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, unix.SO_REUSEADDR|unix.SO_REUSEPORT, 1)
13                        })
14                        return err
15                },
16        }
17
18        conn, err := d.Dial("tcp", "127.0.0.1:10000")
19        if err != nil {
20                panic(err)
21        }
22
23        conn.Write([]byte("foo bar"))
24}
listener
 1func listen() {
 2        lc := &net.ListenConfig{
 3                Control: func(network, address string, c syscall.RawConn) error {
 4                        var err error
 5                        c.Control(func(fd uintptr) {
 6                                err = syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, unix.SO_REUSEADDR|unix.SO_REUSEPORT, 1)
 7                        })
 8                        return err
 9                },
10        }
11
12        l, err := lc.Listen(context.Background(), "tcp", "127.0.0.1:9999")
13        if err != nil {
14                panic(err)
15        }
16
17        for {
18                conn, err := l.Accept()
19                if err != nil {
20                        panic(err)
21                }
22                conn.Write([]byte("hello world"))
23        }
24}