Drupal was chosen for the widespread community support and third party integrations. Drupal's flexibility to build custom content types, or "entities" as they are called, was the major driver behind the choice for Drupal. ShopCircuit uses a flat data storage to manage millions of product records. Drupal offers an Entity API to interact with, read, save and update our data. The product catalog is stored in a custom MongoDb backend, for which the Drupal community offers a MongoDb API.
Furthermore Drupal integrates well with non-core technologies, such as Apache Solr, Memcached and MongoDb. Drupal's granular caching framework, using separate bins for separate content types, is of great use to our project. Because ShopCircuit is a search engine we cannot cache full pages, since there are infinite combinations of results. Therefore, using Drupal's framework, we use EntityCache in order to cache individual entities in Memcached (stored in server RAM).
Drupal allows ShopCircuit to scale by adding servers "buckets". We can:
- move the filesystem to a CDN
- scale caching over Memcache servers
- scale search facets accross a Solr cloud
- scale our big product datasets accross a MongoDb bins
Drupal ties it all together nicely! Agreed, it requires fine-tuning and developer expertise, but the result is scalability with performance.
Interesting fact: about three years ago ShopCircuit started out as a Drupal 6 test project which lived only a few months. The idea for a product search engine on Drupal was almost abandoned end of 2010 due to the inflexibility of Drupal 6's core search feature. But then came along Drupal 7 with Search API (special thanks to drunken monkey!) and I was sold. I am even more excited about the upcoming Drupal 8 with its Symfony framework integration. Drupal is a mature and stable technology, which nevertheless continues to innovate. Drupal's philosophy is to 'break APIs between major releases, but keep your data compatible'. I believe that is the right balance between stability and innovation.
Developed by Morningtime Drupal Agency