Running Powerful Tor Exit nodes

and What I Learned Along the Way.

For those unfamiliar with Tor, Tor provides an anonymous networking layer used by millions of users daily to subvert oppressive regimes and thwart some of the most pervasive forms of corporate surveillance.

How does Tor enable this privacy? Well, in its most basic form Tor is a mixnet; it relies on a group of volunteer nodes run all over the world. When a user connects to a website through Tor they do not connect directly, rather they choose 3 random nodes and build a kind of virtual encrypted tunnel which they route their request through. The server receives the request and sends the response back through this tunnel, if all goes well it reaches the user without their IP address being exposed to the server.

So what does an exit node do? Well, when I said the user chooses three random nodes, that’s not quite the whole story. If the user is accessing the public internet (clearnet), they always pick one Guard node, one middle node (relay), and one exit node. Each type of node plays a different role. Guard nodes are always the first hop in a Tor circuit, they are changed infrequently to prevent malicious actors working out a Tor users IP address. Relays or Middle nodes are nodes that sit in between Guards and exits, to further prevent correlation from dishonest exit and Guards. Exit nodes sit as the last hop in a Tor tunnel that connects to the clearnet, most exits are making thousands of requests to web pages per second and returning them to users.

So what does it mean to run an exit node?

Well, as I said before, the Tor network is run entirely by volunteers, which also means exit nodes are run by volunteers. However, if we look at the network composition of Tor, we can see that on 13/08/2018 there were:

6320 Relay Nodes, and of those relay nodes

842 were Exits,

And 2453 were Guards

So why are only ~13% of the Tor network exit nodes?

And does this have a negative impact on Tor?

The short answers are:

1. Because it’s an involved and somewhat expensive task to run an exit node, and;

2. Yes. Because most users request data from the clearnet, the Tor network can only relay as much bandwidth as its exits provide, which is proportionally much smaller than the bandwidth of relay and Guard nodes. It also makes the Tor network much more vulnerable to sybil attacks at the exit node.

But how involved is this process? Well instead of sitting and observing from the sidelines, I decided that I would setup and run my own exit nodes to assess the process.

Before we go any further, I should mention that the main motivation in writing this guide was to research the difficulty of operating an Exit node so I could provide recommendations not only to the Tor network, but also to the Loki Network — a project I am currently working on that is building a new type of Mixnet (you can read more here.)

Stage 1 — Verification and KYC

I registered accounts with the below VPS providers for one month. I selected these providers based on both popularity and by checking what VPS companies the longest living and highest bandwidth Tor exit nodes used here.

After registration, the majority of the providers used my supplied phone number to call me and verify my identity. Because I registered as a company and not under my individual name, many requested that I show details of company registration. Proof of address was regularly required, which would make registering an exit node anonymously difficult.

If you want to run a Tor exit node anonymously, I recommend you pay with cryptocurrency. None of the providers below accept cryptocurrency, but you can often find resellers who will accept cryptocurrency and will purchase servers for you, this is your best bet for running a node without providing identification.

Stage 2 — Setup

Setup of a Tor exit node on Debian and Ubuntu is covered in detail here. All exits were run on Ubuntu 16.04 or 18.04. The only things I really needed to change in ‘torrc’ were my nickname, and exit policy. It is also recommended (if you are running a high bandwidth exit), that you serve a webpage on port 80 which informs curious users that you are operating a Tor exit. In my experience, this did not reduce the volume of abuse requests as most are automated requests.

I also recommend that you setup Nyx (previously ARM). It’s a great CLI utility for monitoring usage of your exit node, which displays some use statistics on bandwidth consumption and inbound and outbound connections.

The exit policy I used when I began operation for my exit nodes was as follows:

reject 0.0.0.0/8:*reject 169.254.0.0/16:*reject 127.0.0.0/8:*reject 192.168.0.0/16:*reject 10.0.0.0/8:*reject 172.16.0.0/12:*reject 142.93.17.76:*reject 192.99.238.35:*reject *:25reject *:119reject *:135–139reject *:445reject *:563reject *:1214reject *:4661–4666reject *:6346–6429reject *:6699reject *:6881–6999accept *:*

This exit policy blocks some of the largest abuse vectors, though by no means does it prevent abuse requests. I will talk about using a reduced exit policy further in this article.

Stage 3 — Usage Statistics

Ideally, servers operated by the same user should all be grouped into the same family, so paths chosen by users can select a diverse group of nodes. However, the more exit nodes you operate, the more involved the process becomes, having to manually add the fingerprints of each router to each other. So for the purpose of this guide, all of my exit node nicknames had the ‘Knight’ prefix.

When setting up initially, my exit nodes did not use all available bandwidth. In fact, all nodes that join the Tor network must traverse a few tests. which are performed by Tor’s directory authorities (you can read about them in detail here). For the first 3 days my exits only relayed a very small amount of traffic (around 1–2 MiB’s). This is called ‘the untested phase,’ and essentially the directory authorities are testing to see that you aren’t misreporting your bandwidth.

During the next 8 days, my exits slowly ramped up to around 10–20 MiB. At this point, they were actively participating in the network and creating paths. Sometime around the 8 day mark, they got the Guard flag, and after that they could push their full bandwidth.

The image below shows the result of my relays after 20 days. You can see here most of my exit nodes have received the Guard flag which tends to result in a higher usage of bandwidth. The two nodes lacking the Guard flag are missing the flag because they were labelled as offline for a period of time due to abuse takedowns.

As you can see, the results of different VPS providers have varied wildly. There were many that claimed high performance but delivered very low bandwidth numbers. There was also a very clear distinction between the two dedicated servers I rented, and the performance of the nodes.

By far, the best node that I operated was KnightSentence, which was a server rented from Hetzner. This node was, for a period of 2 days, the second highest bandwidth node on the whole Tor network. Although this did not come without its downsides; primarily the fact that Hetzner did not like the amount of abuse requests I was receiving (read below in my VPS review).

At its peak, KnightSentence was relaying 3.2 TB of data (combined upload and download). Hetzner does limit nodes at a rate of 30TB per month of downloads, so this performance didn’t last forever. But Hetzner also allows for a flat rate of €1 for each TB past your cap, so running this monthly at that rate would not incur massive fees.

Stage 4 — Abuse

So now the fun part. As I mentioned before, Tor exit nodes relay requests to the internet, and although the majority of the traffic they relay is honest and legitimate traffic, there is a small percentage of Tor users who choose to perform attacks… and use Tor exit nodes to perform these attacks.

This means that exit node operators have to deal with abuse emails from VPS providers. Over the course of running 10 Tor exit nodes for 30 days, I received 258 abuse emails, with an average of about 9 abuse requests per day. Now that may sound like a large amount, but considering that the bandwidth provided by my nodes was a cumulative ~ 240 MiB’s, the amount of abuse claims I received was comparatively small.

Abuse claims sorted by category:

There was also a direct relationship between how much bandwidth was relayed, and how many abuse requests I received. Most abuse requests were relayed from Hetzner and Leaseweb. Some providers who provided high bandwidth like Scaleway, Netcup and Online.net rarely sent abuse requests, and when they did, they tended to send only serious abuse requests.

Some interesting abuse requests I collected over the month were:

• DMCA Notice for a Torrented episode of Ben 10
• SSH Abuse requests from the FSI (Korean Government Body)
• SSH logins with the username listed in the abuse logs as “FUCKYOUCUNT”

To give you an idea of the average abuse email, I’ve uploaded a full copy of an abuse email here.

All abuse emails were answered by hand. Most VPS providers like Hetzner and Leaseweb now have online portals where you answer all abuse requests. I drew heavily on Tor’s abuse template page which provides a long list of pre written responses for each situation you are likely to encounter, however, a lot of the time I wrote my own personalised responses. I also found some relevant legislation that applies to European VPS providers that I included in my version of a Tor DMCA response for europe which I call a EUCD response template here.

However, sometimes this was not enough to prevent my servers from being shut down. Over the course of the 30 days, I experienced 10 different server shutdown events, and I was blacklisted by more than one VPS provider, which I discuss below.

Stage 5 — Provider Review and Cost and Customer Support

Below are my overall thoughts on running an exit node with each provider I tested over the one month period. I give a detailed review on each provider, including why I chose to use them and how my overall experience was.

OVH is a French VPS provider, and I found OVH to be one of the best exit node providers. Over the 30 day period, I did not receive an abuse request from them that I had to respond to. Additionally, there was no period in which OVH shutdown access to my VPS or temporarily suspended my VPS. Comparatively, OVH provided lower bandwidth than some other providers, but if you’re looking for a low cost and easy to deal with option, OVH would be highly recommended.

Online.net

Online.net is also a French VPS provider which is quite popular on Tor. I received about 10 abuse requests from online.net over the course of my time with them. Answering requests through the portal was easy, and I did not experience any downtime from server takedowns. The bandwidth they provided was comparatively good for the relatively low cost of operation. I also found their sign up process relatively easy, only having to provide a valid phone number.

Scaleway

I had quite a few difficulties with scaleway, although the cost was low for the high levels of bandwidth provided. Most of the difficulties I had were related to their inability to forward abuse requests to my email. Multiple times, my server was suspended for abuse claims that I failed to answer. If you keep up to date with Scaleway, and login each day to check your abuse requests, I don’t believe that you will have a problem, however, keep this in mind if you decide to use Scaleway.

Netcup

Netcup provided a high level of bandwidth for a very low cost. Their user interface for managing your server is quite old and dated, but they do have a method for replying to abuse requests. Although the bandwidth provided by Netcup is good, I found their staff to be quite aggressive when it came to abuse claims. They tend to disable servers while they investigate claims and wait for your response, and this can lead to significant downtime for your servers. They quickly told me to operate my exit node with the reduced exit policy, and I complied, however even after this they still repeatedly banned me, and re-enabled access after I had dealt with abuse claims. I would not recommend running anything more than a middle relay with Netcup, unless you can tolerate high downtime.

Leaseweb

Leaseweb performed well, regularly ranking in the top 3 relays that I operated. It was one of the more expensive servers I rented, costing just as much as some of the dedicated servers I rented. However about a month after I finished my operation with Leaseweb I received a bill for 6500 euros. Turns out Leaseweb does not provide any warnings that you have gone over your allocated bandwidth allocation, and you have to manually set bandwidth allocation caps to make sure you don’t exceed them. I was able to negotiate these excess costs down significantly however i consider their excess bandwidth policy to be somewhat predatory.

Leaseweb provided a relatively smooth user experience, and abuse requests were always sent to my email, which I responded to using their abuse ticket system. Over the course of operation, I experienced no downtime with Leaseweb. I would strongly recommend avoiding leaseweb unless you set strict bandwidth limits on yourself.

Digital Ocean

Digital ocean performed quite poorly considering the cost. They provided a low bandwidth rate of ~8 MiB’s, and although they advertise their servers as costing $15 a month, they charge $0.01 for each GB of excess bandwidth which pushed my bill for about 20 days of use to $37. Overall, I feel like this was not appropriately signaled to me, however, maybe I was just ignoring messages as I was setting up multiple nodes. Digital Ocean sends the most DMCA requests of any server operator in my experience, however, they have no formal way of responding to tickets. Most of the time I just left abuse requests unanswered, and they did not seem to care, as my server continued to operate.

Feral Hosting

Feral Hosting has a strange interface and setup. You essentially buy a ‘slot’ on a server (this is the only server in which I was not given root access), and all software must be installed as a normal user, which can make Tor setup tricky. They are also one of the only providers who explicitly prevent you from running Tor exit nodes in their Terms of Service, so I only ran a Tor middle node here. The experience overall was okay, but I would say running a middle node elsewhere would be cheaper and less hassle if you’re going to install anything custom on your box which requires root permissions.

Trabia

I decided to rent dedicated servers from two providers, and Trabia was one of them, hence the significantly costlier price. For the cost of renting the server, Trabia provided me with a lower bandwidth than I would have expected. Trabia also suspended my server once over the course of testing; they claim it was because I didn’t answer a support ticket. Trabia seems to send less abuse requests than some other providers, but because of the high cost and low bandwidth performance, I could not recommend running a Tor exit here.

Hetzner

Hetzner provided the best VPS by far in overall performance, as it was the node that reached the number two position as the highest bandwidth capacity node running on Tor. However, this was not without its own issues. Over the course of running my node with Hezner, I received more abuse requests than from any other provider. Even proportionally to its high traffic, I received a high number of abuse claims from Hetzner. Hetzner has an online abuse ticketing system which is quite easy to use, and emails were always relayed to my email address.

About 15 days into operation I received an email from Hetzner informing me that I was on a blacklist, that I would not be able to purchase additional services from Hetzner, and that after the period for which I had rented my server, my account would be deactivated. Soon after that, Hetzner suspended my server for receiving too many abuse requests. They said this ban could not be appealed, however I sent them a personalised email explaining my situation and the fact that I was running a powerful Tor exit node. They agreed to unban my server, but told me that I should not continue to run an exit node. Our correspondence continued, and I suggested that I run an exit with the ‘reduced exit policy.’ The Hezner representative said this may be okay, and they would continue to monitor abuse, however if the abuse claims continued at the same rate, they would again have to suspend my service. The representative also suggested that instead of running an exit node, that I run a middle node, which would be acceptable.

Overall, Hetzner was a mixed bag. They don’t seem completely adverse to Tor usage, however they seem to want to remove you from their services once you reach a high amount of abuse requests. Hetzner also enables a 30TB download limit per month, and once your reach that limit you can pay an extra 1 euro per additional TB, otherwise they cap you to 10 Mbps.

Wedos

Their user experience is terrible. They only give you an IPV6 address when you buy your server, and you have to request an IPv4 address by lodging a support ticket. Add to this the fact that they do not allow SSH connection to their boxes, only VNC. I would not recommend them for these reasons.

VPSCheap

Even with polite responses to abuse claims they will ban you for running an exit node, they also provide poor network performance.

What does all this mean?

This whole experience has taught me a fair few things about the Tor network. Here are some observations and suggestions for exit node operators and the Tor project.

1. Running a Tor exit node is a time consuming process.
2. Running nodes that provide a high bandwidth to the Tor network is not as difficult as I initially thought, which is somewhat scary considering the resources of state level actors.
3. VPS providers are more reasonable about the operation of Tor exit nodes than I previously believed. Having conversations with admin staff usually produces good results.
4. Don’t expect your Tor exit node to immediately use all available bandwidth- usage develops over time.
5. Legal templates could be updated to mention relevant european EUCD legislation, which protects Exit node operators. Legal templates should also more explicitly cover abuse requests related to port scanning.
6. A dedicated abuse response website where you could easily generate ticket responses would greatly reduce the time spent answering abuse requests. I am looking to create something like this, for Tor and Lokinet.

I hope this helps other people who are interested in running exit nodes, and the important role that exit nodes play in the preserving of digital rights.