Today, for the first time, we are going to discuss a really, really simple Linux howto: Setting up an automated, headless torrent downloader and streaming media server using:
- Server Environment
- Ubuntu Server 11.04 (AMD64 for preference)
- Full LAMP stack already installed and configured to your liking (technically optional, but come on....)
- Correctly configured local network. (firewall and so on)
- Software to be installed and configured:
- Transmission-daemon
- Dropbox
- Mediatomb
Please note that this tutorial will not cover installation of the server environment nor of the LAMP stack. This tutorial will also assume only the bittorrent client will be accessible from outside your local network; allowing everything to be accessible to the big, bad internetz is not safe. In fact, I strongly advise that you only allow it to be accessible for a short time for debugging purposes, as everything else is handled via Dropbox once it's all set up!
Remember folks, more open to the world = more attack vectors = we're in ur base killin ur d00ds.
This tutorial assumes that you already have an account with Dropbox and it is set up on your desktop/laptop/daily driver. Heck, I use my Android-based phone sometimes.
ALSO NOTE - My plan is to keep this updated, so as server versions change, or changes to the software occur, this tutorial should be pretty much valid.
OK, here's the easy part:
INSTALLING THE SOFTWARE
I recommend you do the following from inside the local network:
everything is done from "/home/<username>/" or "~" directory. If you aren't sure what those are, stop now, get some uber-geek to come over and continue on from here.
SSH onto your server.
copy-and-paste the following into your shell:
"ffmpegthumbnailer" is recommended but completely optional; it sends a thumbnail of each file to the client (in my case a PS3) so you have a pretty picture next to the title.
OK, great. so you have almost everything installed, now to the drop-box part:
copy-and-paste the following into your shell:
For Stable 32-bit:
OR
For Stable 64-bit:
and extract with
then run the dropbox script:
A really annoying repeating message pops up. copy-and-paste the ENTIRE url into a browser on your computer, and sign into Dropbox. OKOKOK, for you pedants out there, I know the dropbox wiki says to not do that, but it seems to work fine. I've done this a couple times with various OSes and network situations. So there. If it doesn't.... read the bottom of this post.
At this point, I would check that you have a folder/directory in your Dropbox folder/directory called something that you will remember. Mine is called "torrents". For the rest of this tutorial, that is the name I will be using for the configuration. Please remember to change this to whatever you named the folder where you are going to temporarily store .torrent files.
Also, here is the one thing that I really, really need you to think about if you wish to proceed:
You will have to make this folder "globally writable". This means that anyone or any process can do pretty much whatever the hell it wants to the contents in this specific folder.
If you ONLY store torrents in there which are about to be processed, then there is very, very little likelihood of any real danger. But this is something you will have to consider and a decision you will have to make on your own. Note also, that this person/process will have to already have access to your server. So really, it's your own fault that this purported person deleted your "Backdoor_Betty_XVI.TEHPWNZER.DVDRIP.xvid.torrent" file; and I, for one, am truly shocked at you.
That being said,
chmod 777 ~/Dropbox/torrents
Dropbox install is finished.
Information courtesy wiki.dropbox.com. For further info: http://wiki.dropbox.com/TipsAndTricks/TextBasedLinuxInstall
CONFIGURATION
OK, this is where things can get confusing, but bear with me...we'll get through this together.
Let's go through this in order of "data in" to "data out" - Dropbox is pretty well handled, so you can get .torrent files to your server. Next, we want to download the torrents and store them somewhere.
Make a place to store your media files.
$mkdir media
Done! well, done-ish. Eventually, I will add a script to this tutorial that automates the organization, but that'll be a few tutorials from now, when I have a chance to show you a couple Bash, perl, or Ruby tricks.
now we are going to edit the transmission-daemon configuration file. This can get scary, but bear with me.
Most tutorials on the web have you edit the wrong file, but here are Fiascoes, Travesties, and Faux Pas, we believe in doing things that bear a fair resemblance to "right". Or at least "close enough"; so it is completely unthinkable that we would be completely wrong. Or at least mildly puzzling. Well, we're not likely wrong, most of the time....whatever.
Transmission-daemon has two configuration files. The permanent, static one and the one that loads at runtime. The default transmission-daemom configuration overwrites the runtime config file for some reason that probably makes sense, but I haven't bothered to look into too far.
The permanent config file is
/etc/transmission-daemon/settings.json
before you go all nimbly-pimbly running from xml-tag to xml-tag, take a second and glance at the documentation:
https://trac.transmissionbt.com/wiki/EditConfigFiles
it's all simple, just scary sounding. Don't panic. Here is my copy, redacted so you don't get all hacky on me.
{
"alt-speed-down": 50,
"alt-speed-enabled": false,
"alt-speed-time-begin": 540,
"alt-speed-time-day": 127,
"alt-speed-time-enabled": false,
"alt-speed-time-end": 1020,
"alt-speed-up": 50,
"bind-address-ipv4": "0.0.0.0",
"bind-address-ipv6": "::",
"blocklist-enabled": true,
"blocklist-url": "http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz",
"cache-size-mb": 2,
"dht-enabled": true,
"download-dir": "/home/username/media",
"download-limit": 100,
"download-limit-enabled": 0,
"encryption": 1,
"idle-seeding-limit": 30,
"idle-seeding-limit-enabled": false,
"incomplete-dir": "/home/username/torrents",
"incomplete-dir-enabled": true,
"lazy-bitfield-enabled": true,
"lpd-enabled": false,
"max-peers-global": 200,
"message-level": 2,
"open-file-limit": 32,
"peer-limit-global": 240,
"peer-limit-per-torrent": 60,
"peer-port": 51413,
"peer-port-random-high": 65535,
"peer-port-random-low": 49152,
"peer-port-random-on-start": false,
"peer-socket-tos": 0,
"pex-enabled": true,
"port-forwarding-enabled": true,
"preallocation": 1,
"ratio-limit": 2,
"ratio-limit-enabled": false,
"rename-partial-files": true,
"rpc-authentication-required": true,
"rpc-bind-address": "0.0.0.0",
"rpc-enabled": true,
"rpc-password": "enter a decent password, it will be changed into a hash after the first run",
"rpc-port": 9091,
"rpc-username": "something sensible here",
"rpc-whitelist": "192.168.*.*,*",
"rpc-whitelist-enabled": true,
"script-torrent-done-enabled": false,
"script-torrent-done-filename": "",
"speed-limit-down": 100,
"speed-limit-down-enabled": false,
"speed-limit-up": 100,
"speed-limit-up-enabled": false,
"start-added-torrents": true,
"trash-original-torrent-files": false,
"umask": 18,
"upload-limit": 100,
"upload-limit-enabled": 0,
"upload-slots-per-torrent": 14,
"watch-dir": "/home/username/Dropbox/torrents/",
"watch-dir-enabled": true
}
change everything that says "username" to YOUR username. not "username" that would be silly.
so ready? breath deep.
make a copy so if you screw up, you can start over by copying it back!
$ sudo cp /etc/transmission-daemon/settings.json /etc/transmission-daemon/settings.json.backup
then edit
$sudo nano /etc/transmission-daemon/settings.json
then we restart the service:
$sudo service transmission-daemon restart
Then we test!
open a browser and go to your server's IP address, port 9091
http://SERVERNAME:9091
You should get something like this:
If not, go through the settings.json file again and reload the service.
If you still have no luck, re-read the documentation. If you STILL have no luck, let me know. Be warned - if this is a simple issue and is clearly one of you not reading the documentation, I reserve the right to make fun of you before I help. I will, however, help....eventually.
MEDIATOMB CONFIGURATION
The default configuration runs on sqlite3. I prefer to store the data as a mysql database, as I haven't the faintest idea how to work with sqlite. (seriously...the heavyweight DBs are so easy to work with, I never bothered to learn anything about sqlite, I know that it works, and works well, and how to connect using the various APIs out there, but that's really it)So the NEXT step is completely optional:
MySQL Configuration
login to your MySQL console as root$ mysql -u root -p
enter password: <---do so. This is the MySQL "root" password, not your login password. They ARE different, right? no? shame.
mysql> CREATE DATABASE mediatomb;
mysql> CREATE USER 'mediatomb'@'localhost';
OK, that last one is weird - we are creating a user with NO password on a live database. Don't worry, this user will only have access to the mediatomb database and only can access MySQL on the server itself and nothing/nowhere else AT ALL. This isn't even a real user! No one can log in on your server as "mediatomb". Relax. Programs do this all the time, and don't tell you. I'm just the honest sort.
mysql> GRANT ALL ON mediatomb.* TO 'mediatomb'@'localhost';
mysql> exit;
We are now ready to edit the actual mediatomb configuration file.
first, make a backup!
$ sudo cp /etc/mediatomb/config.xml /etc/mediatomb/config.xml.backup
now edit
$ sudo nano /etc/mediatomb/config.xml
or vi or ed or whatever editor with which you are most comfortable.
edit your file to be like THIS:
<?xml version="1.0" encoding="UTF-8"?>
<config version="2" xmlns="http://mediatomb.cc/config/2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://mediatomb.cc/config/2 http://mediatomb.cc/config/2.xsd"><!--
Read /usr/share/doc/mediatomb-common/README.gz section 6 for more
information on creating and using config.xml configration files.
-->
<server>
<ui enabled="yes" show-tooltips="yes">
<accounts enabled="no" session-timeout="30">
<account user="your username" password="your password"/>
</accounts>
</ui>
<name>Mediatomb (or whatever you want, really)</name>
<udn>uuid:0ffba0119-99b3-4961-904f-ef60b95791e2</udn>
<home>/var/lib/mediatomb</home>
<webroot>/usr/share/mediatomb/web</webroot>
<port>anything above 49152. I use 51000</port><!--this is added by you-->
<storage caching="yes">
<sqlite3 enabled="no">
<database-file>mediatomb.db</database-file>
</sqlite3>
<mysql enabled="yes">
<host>localhost</host>
<username>mediatomb</username>
<database>mediatomb</database>
</mysql>
</storage>
<protocolInfo extend="yes"/><!-- For PS3 support change to "yes" --><!--
Uncomment the lines below to get rid of jerky avi playback on the
DSM320 or to enable subtitles support on the DSM units
-->
-
<custom-http-headers>
<add header="X-User-Agent: redsonic"/>
</custom-http-headers>
<manufacturerURL>redsonic.com</manufacturerURL>
<modelNumber>105</modelNumber>
<!-- Uncomment the line below if you have a Telegent TG100 --><!--
<upnp-string-limit>101</upnp-string-limit>
-->
<extended-runtime-options>
<ffmpegthumbnailer enabled="yes">
<thumbnail-size>128</thumbnail-size>
<seek-percentage>5</seek-percentage>
<filmstrip-overlay>yes</filmstrip-overlay>
<workaround-bugs>no</workaround-bugs>
</ffmpegthumbnailer>
<mark-played-items enabled="no" suppress-cds-updates="yes">
<string mode="prepend">*</string>
</mark-played-items>
</extended-runtime-options>
</server>
<import hidden-files="no">
<scripting script-charset="UTF-8">
<common-script>/usr/share/mediatomb/js/common.js</common-script>
<playlist-script>/usr/share/mediatomb/js/playlists.js</playlist-script>
<virtual-layout type="builtin">
<import-script>/usr/share/mediatomb/js/import.js</import-script>
<dvd-script>/usr/share/mediatomb/js/import-dvd.js</dvd-script>
</virtual-layout>
</scripting>
<mappings>
<extension-mimetype ignore-unknown="no">
<map from="mp3" to="audio/mpeg"/>
<map from="ogg" to="application/ogg"/>
<map from="asf" to="video/x-ms-asf"/>
<map from="asx" to="video/x-ms-asf"/>
<map from="wma" to="audio/x-ms-wma"/>
<map from="wax" to="audio/x-ms-wax"/>
<map from="wmv" to="video/x-ms-wmv"/>
<map from="wvx" to="video/x-ms-wvx"/>
<map from="wm" to="video/x-ms-wm"/>
<map from="wmx" to="video/x-ms-wmx"/>
<map from="m3u" to="audio/x-mpegurl"/>
<map from="pls" to="audio/x-scpls"/>
<map from="flv" to="video/x-flv"/>
<map from="mkv" to="video/x-matroska"/>
<map from="mka" to="audio/x-matroska"/>
<!-- Uncomment the line below for PS3 divx support -->
<map from="avi" to="video/divx"/>
<!-- Uncomment the line below for D-Link DSM / ZyXEL DMA-1000 -->
<!-- <map from="avi" to="video/avi"/> -->
</extension-mimetype>
<mimetype-upnpclass>
<map from="audio/*" to="object.item.audioItem.musicTrack"/>
<map from="video/*" to="object.item.videoItem"/>
<map from="image/*" to="object.item.imageItem"/>
<map from="application/ogg" to="object.item.audioItem.musicTrack"/>
</mimetype-upnpclass>
<mimetype-contenttype>
<treat mimetype="audio/mpeg" as="mp3"/>
<treat mimetype="application/ogg" as="ogg"/>
<treat mimetype="audio/x-flac" as="flac"/>
<treat mimetype="image/jpeg" as="jpg"/>
<treat mimetype="audio/x-mpegurl" as="playlist"/>
<treat mimetype="audio/x-scpls" as="playlist"/>
<treat mimetype="audio/x-wav" as="pcm"/>
<treat mimetype="audio/L16" as="pcm"/>
<treat mimetype="video/x-msvideo" as="avi"/>
<treat mimetype="video/mp4" as="mp4"/>
<treat mimetype="audio/mp4" as="mp4"/>
<treat mimetype="application/x-iso9660" as="dvd"/>
<treat mimetype="application/x-iso9660-image" as="dvd"/>
<treat mimetype="video/x-matroska" as="mkv"/>
<treat mimetype="audio/x-matroska" as="mka"/>
</mimetype-contenttype>
</mappings>
<online-content><!-- Make sure to setup a transcoding profile for flv -->
<YouTube enabled="no" refresh="28800" update-at-start="no" purge-after="604800" racy-content="exclude" format="flv" hd="no">
<favorites user="mediatomb"/>
<standardfeed feed="most_viewed" time-range="today"/>
<playlists user="mediatomb"/>
<uploads user="mediatomb"/>
<standardfeed feed="recently_featured" time-range="today"/>
</YouTube>
<Weborama enabled="no" refresh="28800" update-at-start="no">
<playlist name="Active" type="playlist" mood="active"/>
<playlist name="Metal" type="playlist">
<filter>
<genres>metal</genres>
</filter>
</playlist>
</Weborama>
<AppleTrailers enabled="no" refresh="43200" update-at-start="no" resolution="640"/>
</online-content>
</import>
<transcoding enabled="no">
<mimetype-profile-mappings>
<transcode mimetype="video/x-flv" using="vlcmpeg"/>
<transcode mimetype="application/ogg" using="vlcmpeg"/>
<transcode mimetype="application/ogg" using="oggflac2raw"/>
<transcode mimetype="audio/x-flac" using="oggflac2raw"/>
</mimetype-profile-mappings>
<profiles>
<profile name="oggflac2raw" enabled="no" type="external">
<mimetype>audio/L16</mimetype>
<accept-url>no</accept-url>
<first-resource>yes</first-resource>
<accept-ogg-theora>no</accept-ogg-theora>
<agent command="ogg123" arguments="-d raw -o byteorder:big -f %out %in"/>
<buffer size="1048576" chunk-size="131072" fill-size="262144"/>
</profile>
<profile name="vlcmpeg" enabled="no" type="external">
<mimetype>video/mpeg</mimetype>
<accept-url>yes</accept-url>
<first-resource>yes</first-resource>
<accept-ogg-theora>yes</accept-ogg-theora>
<agent command="vlc" arguments="-I dummy %in --sout #transcode{venc=ffmpeg,vcodec=mp2v,vb=4096,fps=25,aenc=ffmpeg,acodec=mpga,ab=192,samplerate=44100,channels=2}:standard{access=file,mux=ps,dst=%out} vlc:quit"/>
<buffer size="14400000" chunk-size="512000" fill-size="120000"/>
</profile>
</profiles>
</transcoding>
</config>
So basically, you make sure that you have enabled the UI and put in a sensible username and password, disabled sqlite (sqlite3 is "no"), and enabled MYSQL (MySQL is "yes" with the database and username you selected earlier.) Also, as I have a PS3, I've uncommented the "uncomment this..." stuff relating to PS3. If you chose to install ffmpegthumbnailer earlier, enable that as well. It's pretty straight-forward.
to finish, run
$ sudo service mediatomb restart
Next test and add the file you want to watch -
open any browser and go to your server's IP address, at whatever port you set in <port></port>
From here, you click on "File System", and browse to /home/username/media and click in the
button and select "inotify" that will automagically grab stuff that is added there and add it to the serve list.Finally, we test the whole thing.
Add a LEGAL (pfftmmmphhffchtchortle) torrent media file to your Dropbox "torrents" file on your local machine. wait a second or two.
go to the transmission interface. is it there? awesome.
wait for it to download
go to the mediatomb interface.
is it in the mediatomb database? stupendous
go to your PS3/other streaming client. Do you see a "mediatomb" or whatever you named your server? AWESOME. play it, get some popcorn and brag to your honey how awesome you are with your mad l337 haxxor skillz.
This tutorial is for educational/hobby purposes only. At no time will, either jointly or severally, (I thought that bit up myself. HA! who needs law school?) the author, commenter, passers-by, some guy that sent you a link, the Pope, or any other body, terrestrial or otherwise be held liable for anything in anyway or for any reason. I don't promise this will work for you. I do promise I care, but not much. By following the steps outlined above you have entered into a binding agreement to hold the author blameless and defend the author against any claims (or clams) otherwise. You also promise that you will, if reasonable opportunity presents itself,buy the author one (1) beer of the type of his choosing, unless the author is blotto, pickled, senseless, pissed, snockered, or any other sense of rascally drunk, when a cola may be presented as an acceptable substitution. At no time will water be acceptable...You know what fish do in water, right? everything. Lorem Ipsum etc etc etc. Oh, and the author doesn't condone illegal downloading of anything. So fuck off ICE.
EDIT 10Aug11 - Corrected mysql syntax to reflect user with limited permissions and correct syntax.
ReplyDelete