Recursive SFTP Put

PirateNinja

Hi Icrontic

I have a situation where I am transferring digital assets from record labels to popular social networking sites for monetization and partnership programs.

Anyways, doing so requires a sftp setup like so:

All assets that I need to transfer are on a linux server which I have ssh access too. Command line only.

I need to transfer files from that linux server via sftp (no alternative here) to another server which I believe runs Windows but that doesn't matter much.

Anyways, there about 591 directories and the directory structure must remain intact.

So here is my problem:
How do use sftp commands to transfer the directories? From my understanding I can't do this with put and/or mput. At least I haven't been able to so far.
Who knows how to do this? I have to use sftp, no rsync or alternative.

Otherwise I am going to spend hours typing mkdir xxx, cd xxx, put /aa/bb/*.* etc etc

AlexDeGruven

Can you use scp? Works just like cp, but over an SSH connection (like the FTP SFTP relationship).

With scp, you should be able to use:

scp -r "file location"/* user@target.hostname:/target/dir

shwaip

tar/gz it all, transfer it, unzip

PirateNinja

AlexDeGruven wrote:

Can you use scp? Works just like cp, but over an SSH connection (like the FTP SFTP relationship).

With scp, you should be able to use:

scp -r "file location"/* user@target.hostname:/target/dir

Trying this now, thanks. After authorization nothing is happening ... so it appears, maybe files are being transferred. I don't know how to tell. Here is what I see:

[****@****** ~]$ scp -r /home/****/MSM/* **********@mjvstorage.**************.com:/incoming/audio/201004011423
*****************************************************************************
*                      UNAUTHORIZED ACCESS PROHIBITED                       *
*                                                                           *
* This server is private property.  Unauthorized access to this server will *
* be prosecuted to the fullest extent allowable by law.                     *
*****************************************************************************
****@mjvstorage.**************.com's password:

As per the tar ball, I don't know how to extract one given that I am connecting a Windows host and I only have ftp commands available to me. But if there is a way I can try that too.

shwaip

add the v flag for verbosity (scp -rv blah blah blah)

ardichoke

rsync is also an option.

rsync -rv -e ssh local/directory/ user@host:remote/directory/

in the event that some of your files have already copied (via the scp earlier) rsync will figure out and transfer only what needs to be moved.

shwaip

Yeah, i would have suggested rsync, but...

Who knows how to do this? I have to use sftp, no rsync or alternative.

ardichoke

if scp is working, rsync using the -e ssh tag should also. -e ssh using the same method to transfer files as scp.

PirateNinja

Thank you guys for the advice. I have tried scp and rsync. Do those commands require specific ports open / firewall configuration? I think their server is fairly locked on ports. Currently both rsync and scp are timing out, see verbose output below. I think I am going to have to mkdir for a few hours because I have no control of the windows machine or its network and they are all out for the holiday.

debug1: Authentication succeeded (password).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: Sending command: scp -v -r -d -t /incoming/audio/201004011423/
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
Read from remote host mjvstorage.******.com: Connection reset by peer

ardichoke

They both operate over SSH (scp stands for secure copy, the -e ssh flag for rsync forces it to use ssh). Since most sftp servers operating using ssh and an internal loopback to ftp, it was assumed this would work. It appears though that the SSH server software they are using for Windows isn't configured to work with us. You might be stuck having to write a small shell script to sftp the files over. Normally I'd throw something together for you but at the moment I don't have the time to, sorry.