Results 1 to 11 of 11

  1. Post
    #1

    General interest thread HAProxy vs Envoy

    So I was reading rave reviews about Envoy, and how it's significantly better under load vs Nginx or HAProxy, and identical (limited by the receiving servers) on low load, plus stuff like health checks and whatever (available on HAProxy but not Nginx unless you buy Plus). So I was under the impression I'd be able to get a significant increase in throughput for my website.

    Spent the day wading through installing and configuring it (resources are garbage, by the way) I finally managed to get it to pass through requests to my dev server which points back at the proxy for SQL requests.

    This gave me an opportunity to test both to see which actually performs better, as I've got three identical load balancers all sharing a floating IP with KeepAlived.

    Technically HAProxy was at a slight disadvantage as it was also pointing at some other servers but they shouldn't really have any load. It was also doing health checks which Envoy was not.

    Load balancers are both sitting on one VMHost, with dedicated VMs each, 4 cores available and 8GB RAM. They're sitting on the same RAID1 NVMe SSDs.

    Running one at a time, I fired up loader.io and pointed 6000 clients at them both over 30s.

    Same result.

    Then I pointed 8,000 clients over 30s.

    Basically the same result (slight win for HAProxy).

    So finally I pointed 10,000 clients over 30s to each and got this:
    Envoy:
    Click image for larger version. 

Name:	envoy10000-30-8gb.png 
Views:	26 
Size:	63.6 KB 
ID:	228115

    HAProxy:
    Click image for larger version. 

Name:	haproxy10000-30-8gb.png 
Views:	25 
Size:	54.2 KB 
ID:	228116

    Surprisingly HAProxy beat the pants off Envoy with lower memory usage but neither machines (nor the SQL Servers or web servers) got anywhere near full load on any metric. Looks like it got a bit of a surprise at first then recovered, but Envoy could never get over the initial shock.

    So my result is that I've wasted a day but on a positive note I needn't rewrite all of my configuration files. Next thing for me is to find out why HAProxy isn't burning up all of its resources running 10k users over 20 seconds but getting this:
    Click image for larger version. 

Name:	haproxy10000-20-8gb.png 
Views:	24 
Size:	55.6 KB 
ID:	228117

    There must be a non-server bottleneck elsewhere.

    Definitely non-server bottleneck, knocked it up to 8 cores, then knocked it down to 2GB RAM and 2 cores and the HAProxy node had no change in its capability. It's probably my modem.
    Last edited by suntoucher; 30th January 2020 at 8:56 pm.

  2. Post
    #2
    Interesting read, might also be worth adding what versions of each software you were testing

    tbh i'm surprised people were saying envoy is faster, cause i'd be surprised if much could beat haproxy performance

    the only reason you replace haproxy with envoy is for all the service discovery + the other container/microservices stuff not for performance.

  3. Post
    #3
    Abo wrote:
    Interesting read, might also be worth adding what versions of each software you were testing

    tbh i'm surprised people were saying envoy is faster, cause i'd be surprised if much could beat haproxy performance

    the only reason you replace haproxy with envoy is for all the service discovery + the other container/microservices stuff not for performance.
    Found a few articles, this was one of them:
    https://www.loggly.com/blog/benchmar...aefik-and-alb/

    Envoy is 1.13.0
    HAProxy is 2.0.5

    Just realised how I can pinch out a bit more performance. I'll spin up another pair of HAProxy servers on their own floating and use them for SQL load balancing so that internal SQL requests from my app servers aren't competing with external requests on the one load balancer. Plus an extra level of bonus security as my SQL servers stop being externally available.

  4. Post
    #4
    This is actually pretty interesting shit.

    Do you have a blog? You should post it there too, if only for your own records later on.

  5. Post
    #5
    Xenolightning wrote:
    This is actually pretty interesting shit.

    Do you have a blog? You should post it there too, if only for your own records later on.
    Yeah, I've been stewing over setting up Envoy up for a while now after reading about how good it was but eh,
    \_(ツ)_/, grass is always greener I guess. I was certainly surprised.

    Nah, I've dabbled in blogs years ago but never really had content when I had time to write, and don't have time when I have content. So they're scattered about in Wordpress backups on my storage drives.

    This was just me trying out something new in the hopes of improving my (to be released) website. It's important that it doesn't blow up when it releases due to load. I've pretty much Google taught myself the full end to end of running a service mesh whilst building the site itself over the last three years.

    Running it from home, though, means aside from the upfront hardware costs my overheads are like $5 a month (power) and I've got 32 cores, 64GB RAM spread over two machines. One of which will be going to a friends house to ensure uptime.

    Overheads are excluding the Gb fibre I'd have anyway.

  6. Post
    #6
    Send me your site, I want to see.

  7. Post
    #7
    Privoxy wrote:
    Send me your site, I want to see.
    Nein. I've quit my job and on it full time now so should make my April release target, can see it then.

  8. Post
    #8
    Is it something new or is it like Facebook but, or like Trademe but?

  9. Post
    #9
    _new_ TradeMe


  10. Post
    #10
    It's the new version of Neopets. I call it Neoneopets.

    In Japan I'll call it 新新ペット

    All the pets are 20% cooler than Neopets.
    Last edited by suntoucher; 3rd February 2020 at 10:48 am.

  11. Post
    #11
    Very interesting thread, thank you so much for sharing it with us. Have you found out why is it happening? Theoretically speaking, Envoy should be much better than HAProxy, and your results are really strange. I am right now renting the cheapest dedicated servers, and everything is fine, because it has a very good price and quality, much better than ony other hostings on the market. However, if I would have to choose between Envoy and HAProxy, I would definitely go with HAProxy after I have read this thread. You have done a great research, I guess it's going to be useful for a lot of people like me.
    Last edited by clarity1212; 28th February 2020 at 8:13 am.