I never got that far. I'm not a pro in this field, I just briefly researched it once a few years ago because one of my friends is a day trader and he wanted to know what it would take to do automated/algorithmic trading so I looked into a few things.

I'm not sure how much it will cost but I know for sure it's doable. Anyone can get it and I imagine the price will change depending on the exchange.

Pretty sure for HFT you need to use ultra low latency microwave radio waves which require line of sight from one antenna and radio terminal to the next. Which basically means you have to lease space in a building with that connection available as a service.

Nice, I was thinking bajillions. Thanks again you guys. Im off to do work on this a bit. If everything goes to plan I will buy you guys your favorite sport franchises cuz now Im in the empire business.





edit: LL does that take it out of doable range? Are we talking 5 or 7 figures?

Im getting ahead of myself. Ill worry about that when I get there, the price will probably come down in the years this is going to take me to get anything that resembles a working model.

It is the cheapest of the options available, but not sure exactly how much it would be. It doesn't seem unreasonable if you found the right space it would have to be more than mid-high 4 figures monthly, but again I am not really sure about this.

Thats chill. That sounded like some heavy equipment, had me scared.

I'd start by figuring out what your threshold is for trades per second because that's going to dictate the technology you choose I think. A little bit of Googling shows that in the extreme cases you would be writing code that runs directly on some custom hardware but that seems like overkill.

On the other end of the spectrum you have interpreted languages like Python, Ruby or Javascript. In between you have compiled languages like C, Go or anything that runs on the JVM (Java and Scala are the main 2) which is going to offer you very solid performance.

You need to find a balance that's going to execute quickly, allow you to be productive in writing the actual platform and also expose some way so that end users can easily program their own algorithms as long as they know what to do.

I agree. Your use case isn't a good for with heroku. One big part of that is that the client has already gone to the trouble of finding/hiring you.

$400/month is way more than the minimum.

We've hosted a bunch of stuff there and I don't think our monthly bill has ever gotten close to that.

Thats a great piece of information. Even if I am successful with this, I would have to rebuild no matter what to execute more trades. Ill be the only end user though. The dream here is to have the computer trading on one side of the room and I'll be on the other side with 6 flatscreens and Wall st Journal thinking up strategies. Rebuilding because Im not profiting enough seems like it will be a great problem to have.

Right now Im using your 10k trades a second as a benchmark. Is this possible on something like Python? Id like to be able to use Quantopian as a tool.

Hardcore computation metrics isn't what I normally do. I would first get something that works in whatever language you're the most comfortable in. You could start Googling for terms like "HFT engineering", "HFT programming languages", etc..

Perfect, thanks for putting up with me. Google kinda took the wind out of my sails. Its going to be hard to stay competitive. I need to marinate on this and maybe revise.

edit:Some folks are hacking computer monitors for a speed advantage. I wasn't ready to read that.

I wonder how protective these HFT firms are of their software and infrastructure. If you only care about making a successful algorithm then you might want to consider seeing if you can lease a reputable firm's tools/infrastructure rather than build your own from the ground up.

It doesn't seem unreasonable to pay them like $10k/month if it would cost you $30k/month and a lot of headaches to roll your own center.

That would actually be sweet for a lot of reasons. Im going to look into seeing if thats an option. Building it myself also means Id have to buy new hardware like every other month, which I did not think about. I already miss the dream.

Its a little sad because I always wanted to get into programming and reading that article was my light bulb moment on what to make. Anything I build is going to be low frequency with different strats, which is less appealing. Maybe its for the best. The volume and speed entice me.

lots of money in HFT, no one is solid

Okay, the server is alive and alive.

I was immediately interested in seeing if the Heroku version or the Digital Ocean version goes faster, so I did this silly comparison. Heroku is on the left and DO is on the right. I know this isn't highly scientific.

The first test was a single ping after a series of pings to spin up the Heroku dyno:



This next test was a series of 10 pings:



So, it seems like Heroku really starts to get up to speed when there is a bunch of hits while Digital Ocean appears to slow down. I ran 100 pings and it really didn't do much better than the above.

Interestingly, I ran the comparison on the poker bot page, and Heroku did 7 to 9% faster on single pings, and 2% faster on multi-pings.

I'm not sure why that happens, but I also know that the compiled javascript isn't well-optimized and I do have to upload a faster version.

The server stack for me is:
Arch Linux
Nginx
Ring
Other stuff that's not too interesting.

As far as I can tell, Heroku simply Clojure with Leiningen, which basically a raw wrapper around Ring. I created a .jar file and deployed it since I wasn't really in the mood to download and install Leiningen (not in the official repository), and it didn't seem required. Of course, the compiled .jar has Ring sitting inside of it.

Heroku and I both use Git to deploy. I can automate things with bash, but I haven't really got around to doing it yet since I am going to upload another site on the server.

If Heroku is so horrible, where are the five comparable competitors that exploit this?

Edit:
Eh huge "no way" to web based HFT. Direct connections are needed and every indirection kills. We're talking about a business where modding cables and hardware of any kind to squeeze out <millisecond edges is considered normal. I'm a huge Python fan but I would not invest in a HFT quant if I knew they used Python to do any heavy listing at all.

Don't want to be a destroyer of dreams but I'd say it's very, very unlikely that you have any edge.

If you have a value investing background and are mostly looking for a hobby I'd say a good news aggregation system would be way more useful. Maybe some AI based NLP automated financial reports analysis and comparison tool or something.

https://www.appfog.com and https://www.dotcloud.com are in the same space as heroku.

I don't think there's a ton of competition because the problem hasn't reached critical mass and is partly solved with a little elbow grease. Most people are content with using PHP/mysql on a $5/month shared server where that problem is absolutely solved.

For people who want to use something else on their own server, there's plenty of environment management tools out there with thousands of modules for almost anything you can think of. You just have to assemble the config yourself.

This is because they use aggressive caching with Varnish. If you added a varnish layer to you DO stack you would probably see similar speed increases.

So my parents complained about the weak wireless signal in their home. First idea...just replace the router's antennas and hope that's enough.
Seems like their model has antennas that are not supposed to be replaced (can't be unscrewed it seems)...guess I'll have to open it up and see what one can do :P
On the plus side they seem like total crap so replacing them with 12dBi could do the trick as intended

Nice.

I haven't put much thought into speed optimizations yet. I know I'll use caching on that site since most of it almost never changes. Seems like a lot of people use Nginx to serve static resources. I'm not sure about doing something like this, since (I think) it would require me to break my program up and push up different files instead of a single .jar file. I'm wondering if Jetty does just as well in this area. Rather keep my workflow to the minimum.

Dont worry the dream is still alive. I woke up feeling better about it actually. I still think there is millions of ways to identify small trading windows. In 5-10 years hopefully it becomes more viable. I really feel like having HFT capabilities is the edge in today's game and its still in its infancy.

I think I am just going to build something small with the aid of quantopian with non time dependent strategies. I now see you guys were letting me down softly and I appreciate it.

http://www.cloudflare.com/

Its free (for what you are looking for) and it will cache everything for you (and optionally give you a whole load of extra options like on the fly minification and gzipping). IMO cloudflare is one of the most underrated companies out there.

You just point your DNS records at them, and they do all the rest for you. It will automatically work out which pages are static and which are not and distribute the cached pages over a worldwide CDN.

Dreaming is fine but you are much more likely to get there by becoming world-class good at some combination of math, computer science, game theory, machine learning, computational linguistics, statistics, or even plain software engineering than by dwelling on algorithmic trading. What you're capable of coming up with before you're an expert in one or more of those areas is unlikely to be more than tangentially relevant to a consistently winning scalable algorithmic strategy.

that video is the perfect example of the trend in sites that probably have a good service to offer but are terrible at explaining it. i watched the whole video and looked around their homepage and have only the vaguest idea of what they are actually offering me. your description above was more concrete than their. but what was all that jazz about a community of webmasters banding together like ninjas?