Recursive SFTP Put

PirateNinjaPirateNinja Icrontian
edited April 2010 in Science & Tech
Hi Icrontic :)

I have a situation where I am transferring digital assets from record labels to popular social networking sites for monetization and partnership programs.

Anyways, doing so requires a sftp setup like so:

All assets that I need to transfer are on a linux server which I have ssh access too. Command line only.

I need to transfer files from that linux server via sftp (no alternative here) to another server which I believe runs Windows but that doesn't matter much.

Anyways, there about 591 directories and the directory structure must remain intact.

So here is my problem:
How do use sftp commands to transfer the directories? From my understanding I can't do this with put and/or mput. At least I haven't been able to so far.
Who knows how to do this? I have to use sftp, no rsync or alternative.

Otherwise I am going to spend hours typing mkdir xxx, cd xxx, put /aa/bb/*.* etc etc

Comments

  • AlexDeGruvenAlexDeGruven Wut? Meechigan Icrontian
    edited April 2010
    Can you use scp? Works just like cp, but over an SSH connection (like the FTP SFTP relationship).

    With scp, you should be able to use: 
    scp -r "file location"/* user@target.hostname:/target/dir
  • shwaipshwaip bluffin' with my muffin Icrontian
    edited April 2010
    tar/gz it all, transfer it, unzip
  • PirateNinjaPirateNinja Icrontian
    edited April 2010
    AlexDeGruven wrote:
    Can you use scp? Works just like cp, but over an SSH connection (like the FTP SFTP relationship).

    With scp, you should be able to use: 
    scp -r "file location"/* user@target.hostname:/target/dir

    Trying this now, thanks. After authorization nothing is happening ... so it appears, maybe files are being transferred. I don't know how to tell. Here is what I see: 
    [****@****** ~]$ scp -r /home/****/MSM/* **********@mjvstorage.**************.com:/incoming/audio/201004011423
*****************************************************************************
*                      UNAUTHORIZED ACCESS PROHIBITED                       *
*                                                                           *
* This server is private property.  Unauthorized access to this server will *
* be prosecuted to the fullest extent allowable by law.                     *
*****************************************************************************
****@mjvstorage.**************.com's password:

    As per the tar ball, I don't know how to extract one given that I am connecting a Windows host and I only have ftp commands available to me. But if there is a way I can try that too.
  • shwaipshwaip bluffin' with my muffin Icrontian
    edited April 2010
    add the v flag for verbosity (scp -rv blah blah blah)
  • ardichokeardichoke Icrontian
    edited April 2010
    rsync is also an option.

    rsync -rv -e ssh local/directory/ user@host:remote/directory/

    in the event that some of your files have already copied (via the scp earlier) rsync will figure out and transfer only what needs to be moved.
  • shwaipshwaip bluffin' with my muffin Icrontian
    edited April 2010
    Yeah, i would have suggested rsync, but...
    Who knows how to do this? I have to use sftp, no rsync or alternative.
  • ardichokeardichoke Icrontian
    edited April 2010
    if scp is working, rsync using the -e ssh tag should also. -e ssh using the same method to transfer files as scp.
  • PirateNinjaPirateNinja Icrontian
    edited April 2010
    Thank you guys for the advice. I have tried scp and rsync. Do those commands require specific ports open / firewall configuration? I think their server is fairly locked on ports. Currently both rsync and scp are timing out, see verbose output below. I think I am going to have to mkdir for a few hours because I have no control of the windows machine or its network and they are all out for the holiday.
    debug1: Authentication succeeded (password).
    debug1: channel 0: new [client-session]
    debug1: Entering interactive session.
    debug1: Sending command: scp -v -r -d -t /incoming/audio/201004011423/
    debug1: channel 0: free: client-session, nchannels 1
    debug1: fd 0 clearing O_NONBLOCK
    debug1: fd 1 clearing O_NONBLOCK
    Read from remote host mjvstorage.******.com: Connection reset by peer
  • ardichokeardichoke Icrontian
    edited April 2010
    They both operate over SSH (scp stands for secure copy, the -e ssh flag for rsync forces it to use ssh). Since most sftp servers operating using ssh and an internal loopback to ftp, it was assumed this would work. It appears though that the SSH server software they are using for Windows isn't configured to work with us. You might be stuck having to write a small shell script to sftp the files over. Normally I'd throw something together for you but at the moment I don't have the time to, sorry.
Sign In or Register to comment.