DHT, PEX and Magnet Links Explained

Recently The Pirate Bay confirmed it would shut down its tracker for good, instead encouraging the use of DHT, PEX and magnet links. This move confounded many BitTorrent enthusiasts, who although wishing to adapt, were confronted with hard to grasp terminology and technology. Time for some explaining.

The Pirate Bay’s recent confirmation that they had closed down their tracker since DHT and Peer Exchange have matured enough to take over, was coupled with the news that they had added Magnet links to the site. This news has achieved its aim of stimulating discussion, but has also revealed that there is much confusion over how these technologies work.

The key thing to understand is that nobody is being forced to use Magnet links or trackerless torrents. While these long-standing technologies may prove to be the future, they will co-exist with tracker-enabled torrenting for quite some time. For now, nobody will be forced to immediately change their existing downloading habits, although it may be wise to switch to a BitTorrent client that is compatible with these technologies.

In an attempt to clear some of the mystique surrounding DHT, PEX and Magnet links we will walk through all three briefly, hoping to assure those who’ve become confused earlier this week.




DHT

Using DHT instead of trackers is one of the things The Pirate Bay is now trying to encourage, and torrent downloads that rely solely on this technology are often referred to as “trackerless torrents.” DHT is used to find the IP addresses of peers, mostly in addition to a tracker. It is enabled by default in clients such as uTorrent and Vuze and millions of people are already using it without knowing.

DHT’s function is to find peers who are downloading the same files, but without communicating with a central BitTorrent tracker such as that previously operated by The Pirate Bay.

DHT is by no means a new technology. A version debuted in the BitTorrent client Azureus in May 2005 and an alternative but incompatible version was added to Mainline BitTorrent a month later. There is, however, a plugin available for Azureus Vuze which allows it access to the Mainline DHT network used by uTorrent and other clients.


Peer Exchange (“PEX”)

Peer Exchange is yet another means of finding IP addresses. Rather than acting like a tracker, it leverages the knowledge of peers you are connected to, by asking them in turn for the addresses of peers they are connected to. Although it requires a “kick start”, PEX will often uncover more genuine peers than DHT or a tracker.


Magnet links

Traditionally, .torrent files are downloaded from torrent sites. A torrent client then calculates a torrent hash (a kind of fingerprint) based on the files it relates to, and seeks the addresses of peers from a tracker (or the DHT network) before connecting to those peers and downloading the desired content.

Sites can save on bandwidth by calculating torrent hashes themselves and allowing them to be downloaded instead of .torrent files. Given the torrent hash – passed as a parameter within a Magnet link – clients immediately seek the addresses of peers and connect to them to download first the torrent file, and then the desired content.

It is worth noting that BitTorrent can not ditch the .torrent format entirely and rely solely on Magnet links. The .torrent files hold crucial information that is needed to start the downloading process, and this information has to be available in the swarm.

Pirate Bay links cf. Mininova links: When the Magnet link specification first came out, in January last year it called for a particular format (“base32 encoded”). The links that EZTV, Mininova and ShareReactor have displayed for some time all conform to that original specification. In May of last year the specification was changed, in favor of “hex encoding”, and that is the format of the links being displayed by The Pirate Bay. Torrent clients should accept either format.


Compatible Clients

All the main torrent clients: uTorrent 1.8.5, Vuze 4.3.0.2, BitTorrent 6.3, BitComet 1.16, and Transmission 1.76 (and others) support Peer Exchange and DHT (via a plugin in the case of Vuze). Neither BitComet nor Transmission yet support Magnet links but Transmission is planning to include Magnet link support in the upcoming 1.8 release. Bearing in mind that no site, including The Pirate Bay, has yet abandoned support for traditional torrent files, there is plenty of time for support to be added.

We hope that this article has cleared some of the smoke that was generated by The Pirate Bay’s announcements earlier this week. There is no need to panic, cry or be angry, and it’s not a problem if you’re still confused after reading this article. Torrents will still be available and aside from some extra downloading options thanks to sites that add Magnet links, nothing drastic will change in the near future.

Read more!

Best Android Applications

If you have come to love and depend upon your Android-based phone, you’ve no doubt discovered the wealth of applications on the Android Market. But as with the iPhone App store, not every app is a gem. So you’ve probably found yourself installing and quickly removing plenty of apps from your phone, hoping to find ones that are truly useful.

To help in this quest, I’ve narrowed the field to 10 indispensable applications. These apps serve a number of purposes with little overlap. Some are free and some have a price tag, but all of them are at (or near) the top of their category

1. Dolphin (free)
Dolphin is one of the best mobile Web browsers. Period. It is fast, reliable, has real tabs, gestures, and multi-touch. Although the default Android browser is an okay solution, it simply doesn’t compare, feature for feature, to Dolphin. Dolphin is also much faster than most of the other Android solutions. My big question: Why is there no Chrome for Android?

2. ChompSMS (free)
ChompSMS is the best text client for the Android phone. In fact, this text app is so much better than the default, I’m surprised that the various Android-based phones haven’t switched. One aspect that makes this app so much better than the default is the battery usage. Especially in pre-2.0 releases, the Android text app has a problem with going to sleep, so it is constantly draining battery. By switching to Chomp, you will save your battery.

3. iMusic (free)
iMusic is a tricky one because of what it does: It allows unlimited downloads of MP3s. There is a EULA that insists you agree to remove the song as soon as you have listened to it. Or you can just listen to the song off the Web. Either way, this app will have you happily searching music to your heart’s content.


4. FeedR (free)
FeedR is an RSS reader that works as both an app and a widget. So far, in my quest for the perfect RSS widget, it has proven to be the best yet. FeedR can add preconfigured feeds or you can add your own personal favorites. This app beats the competition with its ability to truly auto-update. Many other feed readers (especially those in the widget category) require a user update to function properly. FeedR handles all updates in the background, with no intervention.

5. Advanced Task Killer ($4.99)
Advanced Task Killer is one of those apps you hope you don’t need, but you sometimes must have. I have noticed on occasion a rare app hanging out in the background unnecessarily. This can lead to a much lower battery life and slower app loading or usage. Advanced Task Killer allows you to kill an application with a single click. This particular app killer includes an ignore list, a widget version, and a long-press to menu feature.

6. Exchange by Touchdown (free demo)
Exchange by Touchdown is one of the best means to Exchange connectivity on the Android phone. Its Exchange connectivity is outstanding, and the support from the developers is just as great. Now this app is a demo and is free. There is a paid version, which does not revert to demo mode (after five days). However, you must install the free version first, as the paid version is only a key to unlock the demo. If you need Exchange connectivity on your Android phone, this is THE tool.

7. ConnectBot (free)
ConnectBot is an open source secure shell client for your Android phone. This client can manage simultaneous ssh connections and copy/paste between apps, and it has shortcuts for ctrl sequences. You may notice on its Market entry that a bug exists for the Hero. The upcoming Hero update will solve this bug, and it will be worth the wait. Now if HTC will push the 2.0 update!

8. Barcode Scanner (free)
Barcode Scanner has quickly become one of my favorites. If you need to find product reviews (or price comparisons) while shopping, all you have to do is scan the bar code with your camera and open up the results. No more having to Google a product and sift through pages for reviews and/or prices.

9. Places Directory (free)
Places Directory is one of those apps you will constantly be thankful you have. With this tool, you can locate (thanks to your GPS) any nearby place, such as a hotel, restaurant, retail location, or movie theatre. Everything is broken into categories, and you can bookmark your favorites. From a places listing, you can instantly dial a location’s phone number or visit its Web site


10. Evernote (free)
Evernote is not just for the iPhone. This handy app lets you keep track of nearly anything — and in many ways. You can keep track of things with notes, photos, recordings, and more. And with an Evernote account, you can keep everything in sync online and on your PC. If you’re looking for a one-stop productivity shop app, this is it

Read more!