Open source is the driving force for more and more innovation in technology and business, and open source is the new standard for application development, as described in our blog post, The Three Waves of Open Source Development.
NGINX is one of the most significant open source projects, powering nearly half of the world’s busiest websites. We’re proud of the many significant contributions from our team, both at work and individually. Open source technology and contribution are core to our values as a company.
Now NGINX has teamed up with our partner Netflix, another driving force in open source, to offer a new implementation of the sendfile(2) system call, for FreeBSD only, which further speeds up TCP data transfers.
sendfile runs fast because it avoids copying file data into a buffer before it’s sent. The new version of
sendfile further speeds up and simplifies large data transfers by supporting asynchronous I/O.
The new sendfile is a product of the ongoing development partnership between NGINX and Netflix, and was released in tandem with Netflix’ recent service expansion to nearly 200 countries. Netflix is using it in production to send multiple tens of gigabytes per second.
The new implementation of
sendfile adds two new flags and redefines an existing flag:
SF_NOCACHEtells the kernel that data is not to be cached after it’s sent.
SF_READAHEADis a macro that allows you to specify the read-ahead size in pages.
SF_NODISKIOno longer blocks, and returns
EBUSYin the rare
situations where the file is temporarily busy. In this case, you can retry the operation in a moment.
sendfile is a drop-in replacement for the previous one. You don’t need to make any changes to your applications. However, you can now expect significantly better performance in cases where the old
sendfile blocked on disk I/O, and you can add the
SF_NOCACHE flag and the
SF_READAHEAD macro, with a page count, to modify the action of
For more information, please see these related links:
- Basic operation of sendfile
- SVN commit message for new sendfile
- Use of sendfile by Netflix
- Thread pool use with sendfile
- Reference documentation for
sendfile_max_chunk(ignore the note about
sendfile bypasses user space, so filters that change content, such as
gzip, do not normally work in combination with
sendfile. To avoid bypassing filters, NGINX automatically disables
sendfile when a configuration context includes both
sendfile and one or more directives that activate a filter which modifies content.
Image appearing for this post on the NGINX, Inc. blog home page is courtesy of NASA.
The post NGINX and Netflix Contribute New sendfile(2) to FreeBSD appeared first on NGINX.