A friend of mine had a specific problem: he needed to transfer a big file (and with big I mean 10GB) from one side of the country to the other. Time to burn a DVD (in fact, split the file into 3 parts and burn 3 DVDs) and send it, was not available. Besides, it is a hassle. I did some basic Google searches on how to transmit big files and found a few services that let you do it, some mailers that allow big files, etc. All of them have a file size limit. Most of them are too low (in the vicinity of 2GB) and almost all of them required registration, fees, and other annoying things. And that in the world of broadband, where I should be able to send my 10GB wedding video in little over one and a half hour. The best one I found was Memeo Send, but to use it more than 3 times, you’ll have to pay.

So I started thinking; how can I create a program to transfer big files?

The first thought is to create an application, that will listen on a TCP port and the same app on the other side will connect. Simple, easy, will work. One problem: everybody is behind a firewall/router these days and the listening side will only hear cybersilence.This made me make a list of demands of the program, will it ever be a succes (meaning, I will use it myself):

  1. Easy-to-use; this means creating a simple user interface and a decent installer (or something that works out-of-the-box like PuTTy).
  2. Works behind a router/firewall; this is the hard part. The app will have to support something like UPnP, or it could use a third party for the communication. I thought about IRC, MSN, or any other protocol that can help here. It should be freely available to minimise any setup-hassle and, of course, costs.
  3. Works on all major OSs; Linux, Windows, Mac OSX, etc.
  4. No size limit. Ever. The only limit should be the receiving end’s free hard disk space.

If anyone has any ideas, please share. If your idea is used, your name will be credited in the program and you will live on forever in the realm of people-that-have-contributed-to-some-computer-program.

Update: the first released version is available..


5 Comments

tobias n · Thursday November 5th, 2009 at 01:02 PM

Use Mikogo to take over someone’s computer and just drag and drop your files from one end to the other…

Bart · Thursday November 5th, 2009 at 03:51 PM

Looks good, but has a few problems:
– File size limit in transfer is 200MB (http://www.mikogo.com/en/product/features/#transfer)
– There is no Linux version
– Mikogo is a third party. If they decide to ask money, you’re stuck again.

bas · Thursday November 12th, 2009 at 10:22 AM

hi bart, nice blog you have!

Not a nice gui solution, but if both sides use linux (or windows cygwin+ssh) and you have no firewall issues (or nat port-forwarding setup on i.e. port 2222)

on the target (listening)~# nc -l -p 2222 | cat > hugefile

on the source (sending)~# cat hugefile | nc target-hostname 2222

pros: works all the time, even with terabytes of data
cons: no encrytion, no gui, needs port forwarding on target.

Another more simple option is to setup a webserver on the source machine and just serve the file via http.

Bart · Friday November 13th, 2009 at 08:51 AM

Thanks Bas,

I am well aware of the possibilities of cat and netcat (actually, I prefer the program ‘socket’). The cons however are too big to make it any success. It *has* to work in a firewalled setup (using UPnP is promising) and it *has* to be Very Easy.

Thanks for the comment,
Bart

MammothCopy version 0.2 | Ik doe er niet aan mee · Wednesday January 13th, 2010 at 09:18 PM

[…] Sending Really Big Files […]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.