It's 2024! Please avoid writing SSH commands like that.
Instead, configure your ~/.ssh/config with LocalForward, RemoteForward, and ProxyJump. This can save you a significant amount of time, especially when using ssh, scp, or rsync to transfer data from a remote server that requires multiple intermediate SSH connections.
e.g:
Host jump-host-1
HostName jump1.example.com
User your_username
IdentityFile ~/.ssh/id_rsa
Host jump-host-2
HostName jump2.example.com
User your_username
IdentityFile ~/.ssh/id_rsa
ProxyJump jump-host-1
Host jump-host-3
HostName jump3.example.com
User your_username
IdentityFile ~/.ssh/id_rsa
ProxyJump jump-host-2
Host target-server
HostName target.example.com
User your_username
IdentityFile ~/.ssh/id_rsa
ProxyJump jump-host-3
LocalForward 0.0.0.0:8080 0.0.0.0:80
RemoteForward 0.0.0.0:9022 0.0.0.0:22
# after this:
# - you can ssh/scp/rsync to your target-server via an alias
# - forward traffic FROM port 80 on your target-server to port 8080 on your local machine
# - forward ssh requests TO port 9022 on your target-server to port 22 on your local machine
# - remember, for LocalForward & RemoteForward :
# + left is target-server
# + right is your local
# + use 0.0.0.0 instead of localhost or 127.0.0.1While we're sharing neat ssh_config tricks, here's my favorite trick I use:
My home network is set up so that if I'm home or on my self-hosted VPN, I can SSH directly to my various things. But if I'm away from home and not on the VPN, I can SSH into my home systems through a jump host.
In the ssh_config file, I have it configured to detect how/where I am and optionally use a jump host.
Host jump jump.example.org
HostName jump.example.org
Port 41444
User mmh
UserKnownHostsFile /dev/null
ChallengeResponseAuthentication no
CheckHostIP no
Compression yes
ForwardX11 no
GSSAPIAuthentication no
LogLevel ERROR
PreferredAuthentications publickey,keyboard-interactive
ProxyJump none
PermitLocalCommand yes
# Order here matters. Detect VPN first, then home network.
# If connecting to a *.example.org host and router.example.org = 10.0.0.1, must be home/vpn.
Match host *.example.org exec "getent ahosts router.example.org | grep -q ^10.0.0.1"
ProxyJump none
# If connecting to a *.example.org host and the macaddr of 10.0.0.1 is NOT 2a:70:ff:ff:ff:ff, then use jump.example.org:
Match host *.example.org exec "! arp -ne 10.0.0.1 | grep -Fq 2a:70:ff:ff:ff:ff"
ProxyJump jump.example.org
## Define the things
Host tv tv.example.org
HostName tv.example.org
User mmhThis is really cool, I didnt know you could use "exec".
You should probably replace that "arp -ne" command with "ip neigh show" instead (assuming this is Linux).
Wow. Nice trick! I didn't know SSH Config can do that exec control flow.
> It's 2024! Please avoid writing SSH commands like that.
Sometimes you just want to do it once and you don't want to write a config file for it.
For example, I am often building up a short lived vps in order to use ssh as a socks proxy and bypass georestrictions.
Also it happens that sometimes you are asked to help a different team and need to access a server you aren't accessing usually and have very few reasons to ever access again.
I think that using 0.0.0.0 it's a bad idea. That is supposedly opening the port in all network interfaces, including the external ones. So, if you don't have a firewall (especially on the remote server) you are exposing something to the world.
OTOH if I'm going to use some tunnelling/port forwarding quite often, I would use the config file option, but for an one time or sporadic use, the command line option is better IMHO.
Nice catch. You're right. At my company all servers operate inside a complex & heavily-guarded intranet, so I usually use 0.0.0.0 instead of localhost / 127.0.0.1. Sometimes, only using the former worked (e.g: using Code-Server or Jupyter Notebook), and I'm not so good at networking to dive into iptables and firewall things.
Speaking of things like this. How are folks managing setup for their local machines to standardize config like this across them? I’ve been in search of a git-ops like solution that is single user/multiple machine compatible doesn’t require having an existing server online somewhere from which configs would be fetched (e.g. I’m fine using GitHub).
But I don’t always want the shell to happen? I whipped up some basic bash/fish functions that let me do it much easier and with minimal arguments to remember, and I can always refer to the well documented functions if I forget something.
This is interesting. I always made a bash function to proxyjump. This seems cleaner. I'll try it.
Oh wow, I was still using `ProxyCommand ssh -W %h:%p jump-host`
SSH tunnelling is an utter necessity in the ridiculous corporate environment I work in. Incredible amounts of bureaucracy and sometimes weeks of waiting to get access to stuff, get ports opened, get some exception in their firewalls and vpn so someone can access a thing they need to do their job.
This guide mentions -D but doesn't really articulate quite how powerful it is if you don't know what it does.
ssh -D 8888 someserver, set your browser's SOCKS proxy to localhost:8888 (firefox still lets you set this without altering system defaults). Now all your browser's traffic is routed via someserver.
I find that to be incredibly useful.
That was pretty much my standard way to browse the web away from home in the mid 2000s. But when I actually got a corporate job they had whitelisted IP addresses so I couldn't even get an SSH connection to some random box on the net. I was so miserable I started to look into setting up http tunnel and somehow getting a box I controlled whitelisted. But instead of going that far I just changed jobs.
It isn't a good idea to circumvent corporate environment networks. they're there for a reason, and doing it shows a lack of professionalism and dis-respect for the organization process, procedures, and security. Yes it takes weeks/months to get access, then it takes weeks/months to get access. You don't want to be held liable for opening a backdoor to confidential information, or compromising their security.
Exactly. It's not a good idea to bypass policies at work. Just because you don't know why the policy is there or you disagree with the reason, it doesn't mean you can ignore the policy.
If you can't get your job done, then escalate the issue to your manager. You not being able to get your work done because of other teams is the kind of problem they're supposed to be solving.
If you let me ssh on that server and I am allowed to ssh from there elsewhere that is not bypassing anything. You allowed me to do that unless it says somewhere that tunnels are not allowed. The question is mainly for which purposes you allowed me to use these things and whether I comply with that. E.g. if I was given a ssh route to reach the some internal LDAP system for software development reasons and I abuse it to stream cat videos on youtube that is on me. But if I use it to reach another internal server that I use for software development, then it is on them.
The alternative would be asking a babysitter for each connection you are making. Sounds like a good way to never get work done.
Also: A good sysadmin will have lines in their /etc/ssh/sshd_config that prevent me from tunneling if they don't want me to do it.
This is the approach I take too. If I need it and I can do it then I'm going to. If you don't want me to then block me.
I must say I've had some raised eyebrows over that approach but if the alternative is not getting my shit done then I'm gonna do it unless explicitly forbidden.
I think that statement is pretty short-sighted.
Bypassing corporate policy at work is risky. You might bring down negative consequences on yourself or your workplace. You have to understand what you are doing. You have to understand likely reactions.
But also, bypassing corporate policy can have benefits. If I'm more productive or get a reputation as the guy who gets things done or don't get seen as a complainer or just generally produce results because I bypassed a policy, those are all benefits. If I can transform "hey boss, it's gonna be another week on this project because I'm waiting on a policy exemption" to "here it is", that's a benefit.
You have to weigh whether the benefits outweigh the risks for you.
Depends on what you mean by bypassing. If it is a workaround that is not prohibited but rather just not known by ICT and most users, there's as good as no personal risk.
If on the other hand, it is sabotaging or disabling safety systems, e.g. exposing the internal network to outside the corporation or writing passwords on a paper lying on your desk, then you can get blamed.
My experience is that this will always be a kind of cat and mouse game and that is just fine. It keeps ICT sharp while there always are possible ways to cut some corners if things need to move forward. Alternatives would mostly be ultimate chaos or crippling bureaucracy.
I do agree, but I'm not sure people are actually thinking about the potential risks. Because it's easy to say "what risk can there possibly be?" but it's hard to actually answer that dismissive question.
Also, the if there is risk analysis it may be overly focused on the short term. I've worked with "here it is" kind of people... and had to deal with the messes they leave behind. Those people get praised in the moment at the expense of the future (some of those cases were actually recognized eventually and the people were let go).
Sometimes they are. Sometimes that reason is long forgotten, or isn't really valid anymore, or is an overprotective measure and not really a good reason in the first place. Quite often it doesn't justify waiting weeks or months to get it changed.
[...] they're there for a reason [...] Yes it takes weeks/months to get access, then it takes weeks/months to get access.
Admittedly, it's a risk to break such rules. One has to be an adult and use good judgement. It's OK, most of the time.
Many corporate networks show a lack of professionalism and a disrespect for the people the network was designed for.
In many corporate cases, SSH tunneling is the desired way of accessing a closed by default port on a firewall. Very often from a predefined bastion host.
If you don't want to open a range of IPs, it allows only people with their ssh key registered on either a selected bastion host or the server to open a specific port.
It can also be a way to authenticate users. For example if you want to secure the access to an open source version of an app for which only the proprietary enterprise tier allow authentication by ldap/AD/oauth2. You can have ssh authenticate against LDAP/AD/oauth2 and leave the app running without authentication enabled or with a single user. As long as you don't need RBAC/privilege separation or some kind of auditing of what each user does on the app this is a particularly valid solution.
I will do everything by the book if your company gives me a person that can help me within half an hour. If every request needs days to complete and then doesn't work and then I have to make another request – if I wouldn't know better I would call it sabotage.
From the CIA simple sabotage field manual: Insist on doing everything through “channels.” Never permit short-cuts to be taken in order to expedite decisions.
New version of https://xkcd.com/303/ ?
"Waiting for corporate to punch a hole through three firewalls for me to get access to the test server :P"
I was on a project once where a consultant had dropped their laptop and it had taken a week or two to get fixed. After that everyone had to use a laptop provided by the client. When we scaled up the project with 3 more developers the project manager who had set up this policy discovered that the lead time for 3 dev laptops meant that the new developers got to be bored for a month at a fairly high hourly rate.
Can you cite any examples of damage resulting from personal browsing over an SSH tunnel that the worker was held liable for?
That is an awfully specific question. Here are a few examples of what could happen though:
- Malicious code on a webpage compromises your computer.
- You download unauthorized software to install, which possibly even comes from a known-bad source.
- Your employer could have trouble establishing that their patent is legitimate because you accessed documentation from a competitor.
Even if the worker avoids liability for costly mistakes, the company will be set back. You can also be fired for breaking rules like that even when there are no actual damages.
The filthiest SSH tunneling hack that I've ever done was at 3AM while in a three-way... datacenter connection. The interesting part of that, while the three facilities, spaced out over a single metro area had upstream transit connectivity to the rest of the net, only two pairs were able to reach the other due to some odd routing policies that weren't able to be resolved in time.
That meant that A could connect to B, and only B could connect to C. The data I had to move from facility A to facility C via B in the most ridiculous rsync+ssh tunnel+keys+routing shenanigan mashup I've ever done. It took a few tries to get the incantation exactly right, but it was magical seeing it all move as one.
Looking back it is super obvious how I'd do it now, but back then being green, was a huge accomplishment. I still remember the exhilaration when I confirmed everything was synced up.
just check my comment in in this post using `~/.ssh/config` with ProxyJump, you can virtually jump between A B C D E ... or whatever.
Yep, had I known that nearly 20 years ago I would have done just that. That's the ideal way to do it now!
If it makes you feel any better, ProxyJump was only added in 2016
I love the extra detail in the visualizations. My wish is for networking to have much more visual representation of traffic, especially at lower level connections.
Hi.. Check out the diagrams here: https://www.nathanhandy.blog/articles/osi-model-revisited.ht... .. obviously this is only a static conceptual representation. Most network vendors will have some form of visual representation of traffic, but it's tyipcally only discreet metrics / graphs.
That is a fantastic example. I will definitely try your HandyDash as well. We should have had the ability to see this detailed traffic breakdown 20 years ago.
Thanks for the feedback. I hope you find HandyDash useful. If you find any issues feel free to raise here: https://github.com/Nathan-Handy/HandyDash/issues
Shameless plug:
What to do when you want to ssh to your linux server or IoT device but they are behind the firewall and without a static IP? You can use a tunneling service like https://sshreach.me.
I've used tunneling quite a lot over the years but never knew about -J option.
What I'd really like is just some visual tool to configure my tunnels instead of spending 30 minutes very few months when I need to use a tunnel.
> TCP-over-TCP
> It lowers the throughput due to more overhead and increases the latency. On connections with packet loss or high latencies (e.x. satellite) it can cause a TCP meltdown.
This actually isn't a problem with SSH tunnels unless you're using TAP/TUN, because It unpacks and forwards the TCP streams. But you can still get reduced performance with multiple channels due to head of line blocking.
> But you can still get reduced performance with multiple channels due to head of line blocking.
Later problem can be solved with the use of pool of separate SSH connections: https://github.com/Snawoot/rsp?tab=readme-ov-file#performanc...
Interesting tech. I've considered implementing something similar but for WebSockets. I'd love to see some data on how effective it is at various levels of packet loss. To guarantee complete avoidance of HoL blocking you would need 1 TCP connection per channel/stream.
I learned how to use ssh tunnels when wanting to bypass a firewall in my university network around 15 years ago, had to change the default port to 443.
Been using it ever since for so much more than just bypassing firewalls.
What purpose have you enjoyed it for beyond bypassing firewalls and exposing local services across a network?
I use it for proxying general internet traffic (such as from your web browser) using the SOCKS5 proxy described in the article. Combined with FoxyProxy or similar it's nice if you want certain traffic (such as to a certain domain which only allows certain IP blocks) to flow from a certain host based on things like the domain.
In essence it is what you mentioned, these are a few practical uses:
- Streaming region locked content from overseas.
- Permanent reverse-tunnel for remote-access with autossh.
- Increased security compared to making services visible to the internet.
- Downloading scientific articles using my university's connection as a proxy.
sshuttle go have a read much nicer for tunnelling... sshuttle -r user@host 10.0.0.0/8
Anything on 10/8 automatic tunnel it's pretty much a vpn over ssh
Kind of related, but I was wondering if there is some kind of redirect functionality in SSH itself. Something like:
- A wants to SSH into B
- B tells A that it must connect to C instead
- A transparently connects to C directly
- B is not a part of the critical data path anymore
Does something like this exist?
I think you could do that with a virtual IP. For some reason my firewall/router doesn't communicate DHCP option 67 correctly, it sends its own address no matter what I do so I had to set up a a virtual IP/rule to route all PXE boot traffic on whatever port that is going to the routers IP, over to the real PXE boot server instead.
It would be misleading if A doesn't know that the real target is C.
Otherwise you can use jump functionality
From A:
ssh -J B C
B could port forward (as in route packets?) to C, but I don't think there are any HTTP Permanent Redirect equivalents, no.
Maybe you can explain the problem more and perhaps there's a more suitable solution?
If you have a host that's somewhat embedded, you can have DNS handle the "routing" for you. You will have to handle fingerprint verification.
I've found VS Code can setup port forwarding tunnels if you remote into a host and its been very useful. Its graphical, no command line incantations to remember and I usually have it running anyways.
Also setting up code tunnels on the workstations I use regularly saves me a lot of headaches.
Sure it's a crutch, ugly and probably unsafe but I'm not a networking guy and need to get my actual stuff done.
It should be pretty safe if you’re using public/private keys (and passwords turned off), keeping all sshd along the path updated, and guarding them like a troll, otherwise it’s a bad choice
See also:
A Visual Guide to SSH Tunnels: Local and Remote Port Forwarding https://iximiuz.com/en/posts/ssh-tunnels/
Bookmarked, thank you.
SSH tunnels are an excellent tool, but nowadays you often want TLS and reverse proxy functionality built in. I maintain a list of such tools here: https://github.com/anderspitman/awesome-tunneling
Your suggested solution Cloudflare Tunnels man in the middles the traffic and it’s not an end to end tunnel. It’s a tunnel to Cloudflare! The users should be warned about this!