The biggest downside, especially for engineering, is the career growth aspect and the ability to move fast towards technological advancements.
There is also a long way to become truly agile especially now that the organization has grown to a decent size.
The expectation of people should work extra hours similar to being in a start-up does not really gel well with everything else that is not so start-up like.
The other downside has also been the location as the company is growing and it does not help much in hiring local talent.