Can't SCP/SFTP to local Linux servers on wireless
 

This is a truly bizarre problem, but I have spent a lot of time testing it and it's 100% repeatable.

The servers are Debian Linux, on a 192.168.100.x address. These are long-running servers which I have connected to often in the past, but have never been able to connect to with my new MBP. This still works with Transmit on other Macs.

The problem only happens when my computer is inside the network, on the wireless network. It works fine when wired, or over a VPN or with a direct outside IP address from any remote location even when using wireless in those locations.

I have tried using the IP addresses and the names.

The symptom is that SCP, SFTP, or Transmit will stay in "connecting" mode for a very long time (around 15 minutes I think) then fail with a "permission denied" error. However, if I enter invalid login data, it fails immediately with the same error. Only if the login details are correct, does it hang for a long time.

Freaky.

Can you show us /etc/sshd_config (or actually on Linux boxes, it's typically found at /etc/ssh/sshd_config) on the ssh server?

Also, can you copy/paste a transcript of a connection attempt to one while using the -v flag (so for example ssh -v adminuser15@192.168.10.102)?

Trevor

Note also that it is sometimes useful to turn on debugging on the server as well. I recall one case where I couldn't figure out what the incompatibility was until I did that.

You should of course try the usual things like lowering the MTU (more often a problem on wireless networks) and disabling IPv6.

A last resort would be to look at the packets using Ethereal.

Interesting thoughts on IPv6 and MTU. I made no changes to them to start with, but worth a try. I won't be at the site until Wednesday though.

Here's sshd_config:

I should be clear that SSH works perfectly. It's only SFTP and SCP that fail. I won't be able to get a connection log for those until I'm at the site again though.

So here's what the verbose switch gives me. First it does this then hangs:
It stays there for a long time, probably 15 minutes, then:
Seems meaningless to me.

Probably not the issue, but 'man scp' on my 10.4.8 machine says nothing about a "-t" option.

Yeah, I don't know what that is, but it was NOT on my command line. The command line is:

scp -v localfile server:/remotedir

I looked at the source code for 'scp' (Darwin10.4.8.ppc/OpenSSH-57.3/openssh/scp.c) and as far as I can understand, what is happening is that it has gone into a client-server mode where it issues commands to the server (the remote sshd) and the "-t" we see in the debug output is one of the flags that is only used internally (in this mode) - it means "to" and this flag is triggered by the detection of a colon in the 2nd arg to 'scp'
(your 2nd arg was "server:/remotedir") but there is no colon in the first argument ("localfile").
I.e. it uses the "-t" when the transfer is from a local file to a remote directory.

The debug log shows:
This indicates that the remote directory (the target of the transfer) is "/etc/asterisk".
It is sending this command to the remote server.
It seems from the source code that it is expecting a response from the server before it will start to transmit the bytes of the source file.
The relevant source code looks like this:
The 'do_cmd' in the above is what is sending the above "scp -v -t /etc/asterisk" command.
The later call to 'response()' is what is waiting for a response from the server.
It is the final line that invokes 'source(1, argv + i)' that is responsible for sending the bytes of the source file to the server. ('argv' is pointing to your "localfile")

If I'm understanding it correctly, the 'source' function should start off by printing a debug message "Sending file modes:"
It should also show a "progress meter" of some sort if that option has been used.
So it looks like in your case it isn't getting into the 'source' function because it is waiting for a response to the initial 'scp' command from the server.

Very interesting overview, thanks. I know nothing about programming, but you really helped me understand what it's doing and actually understand the code.

What's truly bizarre is that this is obviously a networking issue with this specific computer. Well, I say that because of these facts:

Only happens on this laptop
Only happens on wireless connection inside the network
Doesn't happen when wired
Doesn't happen on wireless outside the network via VPN or directly over the internet

It is also happening with Transmit, which I assume uses its own code and not the underlying Unix programs (I could be wrong).

While networking is my expertise, I can't picture what would happen to ONLY SCP/SFTP traffic between only this computer and Linux servers.

It is indeed difficult to conjure up a scenario that would fit the observations.
Is there anything "interesting" about the name of this laptop by any chance?
Does the problem occur with more than one Linux server?
Can you do scp in the reverse direction (transfer a file from the Linux machine to the laptop)?

It might be interesting to try to capture the packets with Ethereal and see what the difference is in the cases where it works (e.g Ethernet) versus those when it doesn't (Airport). I don't know if Ethereal can (intelligibly) show the packets of an SSH-exchange though - maybe it has a man-in-the-middle mode that can do this?

It would also be interesting (and probably rather easier) to turn on maximum debugging in the Linux server's sshd. I recall doing this once and I think I had to edit the xinit config file or something (to add the appropriate "-d" or whatever it is). That should show you what the server receives (e.g. the above "scp -v -t /etc/asterisk" command) and what it does in response.

By the way, you could find out if Transmit is using the 'scp' utility (or whatever) by running 'sudo fs_usage | grep Transmit' in a Terminal window while running Transmit.

Good questions.

The name is Carlos-MBP17. I'm really creative. The problem happens with two Debian-based servers. I haven't tried a connection in reverse, but that's a great idea. The Ethereal idea is good too. Hopefully I'll get time to try it tomorrow.

(Just a stab in the dark) Do you have anything abnormal in the login scripts for your user on the server to which you are connecting?

I think this is a very good idea as to a possible cause.
But it would seem that the shell "dot" files (what I assume you mean by "login scripts") would have to contain something which depended on the network connection somehow. How else would this explain the observation that everything works when connected with Ethernet but not with Airport?

Nothing has been done with login scripts on this machine.

Yup...
I've seen .profiles contain things like using X11's resize (to set COLS/ROWS) which query the xterm being used; if you're logging in remotely, this'll be over the network and if DISPLAY isn't right, strange things can happen. When things aren't quite right, this will cause login delays (though not anywhere near 15 minutes). Why it would do it on one kind of interface but not the other is a very strange issue, but as I said, it was just a stab...

I was researching this issue since it's still plaguing me, and this thread is the only related link in Google...

I've confirmed this to happen with a dozen machines now, running Debian, Ubuntu, CentOS, and others. It happens with a default bare config. The symptoms are always identical; happens only on wireless and with local servers only.

I'm having the exact same problem, with a fortune printed to the console every time. I do indeed have fortune in my logon scripts, so at least this is a diagnostic tidbit that they are getting executed. My -v output (after authorization) is:

Like you I can log on via ssh fine, and on the linux box use scp to copy a file over from the mac, just no scp from the mac to debian directly.

Interesting. I thought I was the only one. Are you on wireless also? Does it work if you are wired?

It gets more interesting...this works fine using a new Airport Extreme, but doesn't work on any other brand of wireless AP that we have tried. It happens to all Core2 type machines with the N wireless card. Time to file a bug I suppose.

There are two common causes of this sort of behaviour.
1/ Shell scripts which confuse SCP.
2/ More probable. MTU discovery is not working. The link can be
fixed by setting the maximum packet length on the link lower.

Same problem: Cause was .tcsh/iterm startup script
 

I had the same problem, and I found the cause was in my .tcshrc startup scripts. In particular, I was using a script that I think was supplied with iTerm.app or that I found by reading someone else's webpage about configuring iTerm.app. The script sets the title bar in iTerm.app. I changed my .tcshrc to only call the script when in iTerm by doing the following:

if ($TERM_PROGRAM == "iTerm.app") then
source ~/.tcsh/iterm
endif

I hope this helps someone.

Well, I never did figure out what was causing the problem for me, but I did a full Linux re-install over the weekend and now it works fine. So it must have been something wrong with the Linux config (for what that's worth).

I encountered the same problem and discovered I had an 'echo' (used for setting titles in screen) on the remote server that was causing the problem. In .bashrc I had: echo -ne "\ek`hostname`\e\\" ... when this line was removed, scp worked again to my iBook over wireless.

I had the same problem with my MacBook Pro. I just commented the lines echoing the window title in the .tcshrc file of the remote server and the problem was solved.

Regards