doesn't there have to be a NAT64 gateway at the router?

There has to be one somewhere. In my case it's running on my router, but your ISP can do it at their end, and since you can use any /96 including public ones, it's also possible for anyone to run one:

  $ dig A +short github.com
  140.82.112.4
  $ dig AAAA +short github.com
  64:ff9b::8c52:7004
  $ dig AAAA +short github.com @2a01:4f8:c2c:123f::1
  2a01:4f8:c2c:123f:64:5:8c52:7004
  2a00:1098:2c::5:8c52:7004
  2a00:1098:2b::1:8c52:7004

Note the embedded 140.82.112.4 in the last four bytes of the v6 addresses, which you can write in v4 format if you want:

  $ getent ahosts 2a00:1098:2b::1:140.82.112.4
  2a00:1098:2b::1:8c52:7004 STREAM 2a00:1098:2b::1:140.82.112.4