Skip to main content

Netflix Technology

I attended a meetup at Netflix last evening titled “Polyglot Persistence at Netflix”, The Cloud Development Engineering (CDE) team presented various aspects of building and maintaining a highly distributed system to meet its ever-growing customer needs. There are almost 160 million users and with its growing popularity of streaming movies and TV shows (many are produced now by Netflix), the demand on its systems is growing rapidly. Polyglot implies the coexistence of many databases and associated software systems.

Netflix cloud platform forms a layer of services, tools, frameworks and technologies that run on top of AWS EC2 in order to implement an efficient and nimble (fast reacting), highly available, globally distributed, scalable and performant solution. They switched over to AWS Cloud over a seven-year period starting back in 2009. It uses Amazon’s RDS and DynamoDB besides S3 for lower cost storage. The front-end is Node.js while the backend uses Java, Python, and Javascript. The team also described how they are using SSD (solid state device) besides memory cache. The main thrust of the evening talk was their use of Cassandra as the distributed database solution.

Apache Cassandra was originally developed at Facebook as a free, open-source, highly scalable, high performance distributed database to handle large amounts of data across many servers with no single point of failure. This global network of storage servers caches content locally to where it will be viewed. This local caching reduces bandwidth costs, reduces latency, and makes it easier to scale the service over a wide area, in this case globally. Here are the key reasons Netflix is a major user of Cassandra (besides others like eBay, Apple, Comcast, Instagram and Reddit):

  • Very large production deployment – 2500 nodes, 420 TB, over 1 Trillion user requests per day. Cassandra is a NOSQL, distributed, document-oriented database that scales horizontally and dynamically as more servers are added without need to re-shard or reboot.
  • Strong write performance with no network performance bottleneck.
  • It’s data model is highly flexible. A sparse 2-dimensional “super column family” architecture allows for rich data model representation (and better performance) beyond just key-value lookup.
  • It’s geographic capabilities – single global cluster can simultaneously replicate data asynchronously as well as service applications across multiple locations. Th eteam last evening showed how users can seamlessly switch over to another data center if failure occurs. Cassandra has been a good choice for cross-data center and cross-regional deployment as customisable replication helps determine which cluster nodes to designate as replicas.

Like Youtube, Netflix has been growing its global reach and customers in providing streaming contents. The key success factor is the database technology to enable such high scale and performance. Other databases like RDS, DynamoDB and MySQL are providing varieties of function such as analytics and metadata store. One impressive part of last evening’s presentation was how they repair any damage to data on the fly by embedding it into the database itself.

from: Jnan Dash’s Weblog
via Jnan Dash

Source: Netflix Technology Via Business Advice.

Comments

Post a Comment

Popular posts from this blog

7 Ways to Grow Your SaaS Startup Faster

Every startup looks for ways to catapult a business to success. Here are some tips for accelerating growth for SaaS (software as a service) companies. 1. Start charging early SaaS startups are often hesitant about charging customers. They think that their product is not yet ready, that it’s necessary to get traction and focus on expanding their customer base rather than on growing revenue. Deep inside, however, they often don’t just have enough faith in their product and are not sure if someone will buy it at all. They prefer keeping hundreds or thousands of free users to attempting to win a few serious customers. Big numbers are comforting but the product remains an unverified idea. Building a product should imply increasing revenue. So don’t hesitate too much—put it to the test by charging and see if it works in the real conditions. This is one way to minimize your risk too. Instead of waiting to launch until you’ve invested huge amounts of time and money, launch early. If...
Post a Comment
Read more

How to Add Web Push Notification to Your WordPress Site

Do you want to add push notifications to your WordPress site? Push notifications allow you to send notifications to users even when they are not visiting your website. In this article, we will show you how to easily add web push notifications to your WordPress site. We will also talk about the best WordPress push notification plugins and how to send desktop & mobile push notifications from your WordPress site. What is Push Notification? Push notifications are clickable messages displayed on top of user’s desktop or notification area on their mobile device. They can be shown even when the user’s browser is not open. Aside from desktop, web push notifications also work on mobile devices. This allows you to reach your users across devices with latest updates and offers. Web push notifications have proven to be a very effective way to convert website visitors into customers and loyal followers. Why Add Web Push Notifications to Your WordPress Site? We have already discussed tha...
Post a Comment
Read more

7 Best WordPress Job Board Plugins and Themes

Are you looking for the best WordPress job board plugin? There are several WordPress job board plugins that you can use to easily create a job board and charge other companies to post jobs. It is one of the ways to make money from your blog , and you can even use it to post your own job openings. In this article, we have picked the best WordPress job board plugins and themes that you can use. Building a Job Board Website with WordPress WordPress is currently one of the most popular website builders in the market, powering more than 30% of all websites on the internet. It can be used to build almost any kind of website including a job board website. There are two types of WordPress websites which often confuses new users. First, there is WordPress.com which is a hosted solution. Second, you have WordPress.org also called self-hosted WordPress. See our article on WordPress.com vs WordPress.org for a side-by-side comparison. We recommend using self-hosted WordPress.org because it g...
Post a Comment
Read more