The Result
So, my scanner has now pinged the whole IPv4 space.
A little quick primer on the latest state of the project:
In the beginning, I wanted to use this project as a search engine. But I soon ran into a good chunk of problems, that made me decide it was out of scope for this project, so I removed the search engine part of the project, and just turned it into a scanner.
The scanner’s main goal is to:
- Find every IP that responds with a success to a ping with a timeout of 32ms.
- Find out if it accepts TCP connections.
- Discover the setup/implementation of the web server behind the IP, should it have a web server.
Those three goals are relatively simple. But the implementation of them differ in complexity, depending on how fast you want this done. The TLDR of this, is that I run every step in parallel.
Step 1, 2 and 3 executes independently of each other. The only thing binding them together is a few ConcurrentQueue.
This works really well. Like insanely fast. My server hoster also thought this was really awesome. So awesome in fact that they send me a little email, congratulating on my achievement.
Yeah… So it turns out, if you want to scan the whole internet in a resonable time, you have to create quite a lot of connections. It was step 2. that gave me the Hetzner achievement. Sending pings (step 1.) with no payload is so cheap that the 8k pings/second didn’t even make Hetzner bat an eye.
But 100 TCP connections/second for 2 seconds in step 2. was enough for the hoster to ask me to stop.
Looking back at it, I see how it might have looked for Hetzner, so I have decided to shelve the project for now.