Quarterly update on the business of solvers commerical and open source
This project is maintained by fhk
Q1 2022
A Quarterly update on the Mathematical Optimization Solver Industry and Ecosystem.
Exploring the creation, usage and adoption of solvers, soft/middle-ware, bespoke or industry specifc solutions.
So this is purely an idea at the moment. If 10 (ten) people sign up I will write the next one and make some improvements to the content :) Or perhaps people will just make PRs and this can continue to live… Or perhaps this is better on some kind of mailing list?
Due to popular demand there is now a slack channel!
or-chat.slack.com
keywords: launch, v0, PoC, optimization, solvers
Here is a first pass at creating some content around the industry and perhaps some valuable discussion. I toyed with the idea of calling the page “not just solvers” after the well known not just bikes youtube channel.
The structure and direction of this is very much in flux and current represents my personal view. But should there be interest in a specific area please feel free to email me.
One of the areas I feel is neglected is the “commercial” or “business” part of the optimization industry.
How are companies evaluating and precuring solvers or software, building teams or working with vendors?
To get a better idea we want to understand the market size and key dynamics.
The only report I have been able to find is this one from 2015. Gartner does a good job of finding the right companies and doing some analysis. Whilst the market estimate is over $100 million USD. I think this falls short.
Also the key ingredient price and licensing are not covered in detail. This has been one of the biggest changes in recent years. On top of this there is also now more fragmentation in the market with users adopting open source solvers and building their own for their usecase. Plus a number of new market entrants. One thing to note is there hasn’t been a level of development on the scale of Tensorflow/Pytorch in open source. This might be possible in the future, perhaps there might even be unification of these AI/ML frameworks and optimzation.
Back to the $$$, the total spend in my mind is over a Billion dollars. Why? It’s not just the license cost, you have to build the tooling and integrations around it. Not to mention the userbase for such solutions. Hopefully, some of the future discussions can shed more light onto what the actual market spend is. Oh and the hardware to run it all! There are even companies building custom hardware with optimization baked in.
The creation of a index. Why do we need a index? The larger field of “data science” is murky at best.
We’d like to know how the adoption of “mathematical optimization solvers” is going in industry. The overlap between academic and commerical usecases aswell as how the open source software is evolving.
The origin of practioners at large most have PhDs and connections to research. But metrics like number of papers/publications seem a little afar from what we are seeking to asses.
Lets start with the number of attendees for INFORMS, in this case over 6.5K individuals. How does this compare to say the large cloud conferences? AWS ~60K, Google ~26K
Starting with the number or roles advertised/aggregated on Linkedin in the United States for “Operations Research” in the last month.
There are 47,448 this may sound like alot but when we compare to “Deep Learning” at 40,413 it becomes even more clear there is a renewed focus.
Both of which can be encopassed in “Data Science” with 161,544 and “Software Engineer” 174,176.
This might come as a surprise as typically Mathematical Optimization has been somewhat niche.
Lets looks for content and new ventures/acquistions.
Google Trends doesnt seem like the right tool to gauge the market interest as when we look for the above areas not much registers.
So lets try news publications using Google Search.
This yields ~22,300 results. Whereas “deep learning” returns 528,000 results. Business analytics comes in at 137,000.
Given this can we also look at how the $$$ are flowing, are there many funding rounds? acquistions? partnership deals? This is left open for further discussion. Though a typical approach can be “solver inside” like o9 solutions + Gurobi. One idea would be to construct a timeline from the orgination of simplex.
Most recently I’ve had numerous conversations along the following lines:
Before we takle these topics lets figure out if it even matters!
Here @Peter Cacioppi delivers what is in recent times to most expressive answer!
Further there is an ongoing dicussion on HackerNews - here.
This provides some great history and the discussion and listing of solvers is valuable.
To better inform the decisions lets also look at:
I will update this as I can, please feel free to add via GitHub PR.
There are a few SaaS like solutions and this is an interesting space in that it would be useful to remove the complexity of Software Engineering and DevOps from the modeling and PoC process. The current status quo is to perhaps use a notebook and then “productionize” these solutions maybe challenge that.
Aftermarket Analytics - Rapid Deployment App Builder
Many of the below server license companies have cloud offerings too.
The only “cloud” focussed company I have come across is NextMv
Local Solver* Also has API for Cloud compute via SDK
Unique product with favourable licensing for deployment “Golden Disk” pay once.
Many flavours including hosted full fledged ?app?
I like the $200 per month developer license.
The default choice but can be expensive
A leading open source project just starting to ramp up, you can help fund it too!
Unique model of open source with commerical model, great for custom things
New on the block and has a nice Python SDK
Gaining traction and has a interesting idea of a reformulator
Great examples plus docs, new solver recently and support via email list is great.
Gurobi has a option to share a license across multiple machines
This is important as many of the commercial solvers restrict this.
You can get better solutions faster in theory. But it can cost \($,\)$.
One of the most current workflows is to compare against the benchmarks.
This lost traction due to Gurobi dominance but seems to be making a resurgence.
http://plato.asu.edu/bench.html
https://mattmilten.github.io/mittelmann-plots/
Based on expereince different companies have different capacity and experience working with clients at different scales.
These number work for both solver companies and buyers.
1 - 10
11 - 50
51 - 200
201 +
There is a dedicated stackexchange for OR - https://or.stackexchange.com/
New slack channel! or-chat.slack.com
Recent posts on HackerNews also show some momentum.
???Future???
???Future???
I’m going to cheat this time and share a WASM post I wrote and presented some time ago. Why? I really think that client side solving will become something!
https://gist.github.com/fhk/0aba10a289a5e6b253fb2abf268469ac
Further there are many more projects in the works.
https://github.com/centrifuge/clp-wasm
Perhaps this year we will see a commerical solver on WASM in the browser or desktop!?
Next up I might share a complete C++ example using or-tools as I also believe that creating binaries is very valuable to help with many parts of the software process.
#TODO
#TODO