Jellyfin doesn’t need any particular setup to work directly from LAN because it doesn’t ever try to use a central login provider the way Plex does.
The only reason OP is struggling with it is because they set it up so that they can only connect to it via Tailscale.
For real