Black Friday Madness – Musings of an E-Commerce Developer
As I sit here typing this post, somewhere close to 30 million Americans are pushing, shoving and otherwise cramming themselves into retail stores after a day of gluttony to partake in the ritual of Black Friday. It’s the one magical day where many retailers experience the largest profits of the year and offer some great deals in order to get more shoppers into their stores for the frenzy of frivolity. Whether it’s brick and mortar stores or their online counterparts, the overarching goal is a common one: throughput. The premise here is quite simple in that the more people you can get into your store looking at merchandise and the more checkout lanes you have open, the more customers you can process in a given period of time and the more money you can potentially make.
For your local retail stores, this means many things. Product placement is critical, with highly popular products usually placed deeper in the store so that you are at increased odds for more impulse purchases of fantastic deals on items you normally wouldn’t buy at all. Also, it’s important for stores to have a flow that can accommodate a mass of customers desperate for retail therapy. Aisles that contain popular products are wide and a conduit that circumnavigates the store is kept clear of debris at all times. I personally think it would be an interesting and amusing application for the Ford-Fulkerson algorithm to maximize the flow of customers through a store, placing higher weights on aisles with more tempting goodies or doorbuster sales.
The Reckoning Approaches
Online stores take a different approach to throughput. For these virtual storefronts, the more people that can hit their sites translate into more sales. That means they need to be able to support intense loads at key sale times. So how can developers for online retailers prepare their sites for the digital onslaught of the Black Friday/Cyber Monday one-two punch? For this article I will be focusing on the use of the new async and await keywords as a way to improve throughput on websites. The async and await keywords are part of the Task-based Asynchronous Pattern (TAP) that was introduced with the release of C# 5.0. It’s not that async/await allow you to do things that weren’t possible before. Asynchronous patterns in .NET have existed since framework version 2.0 with the Asynchronous Programming Model (APM) that brought forth methods like BeginInvoke/EndInvoke and the infamous IAsyncResult. These new keywords just make it much easier to implement and make the code involved much more maintainable as a result. I also want to make it clear that async/await is not a magic bullet and is only part of the total solution. Other throughput improving techniques like output caching and load balancing are still just as important in squeezing the most juice out of your servers.
Let’s analyze a typical ASP.NET web request process. When a page request is received, a thread is dispatched to handle that request. The thread is held while the server processes and builds the response, and once the response has been returned to the client, the thread is returned to the pool to be used for another request. It’s mind numbingly simple, but there are some gotchas. A worker process has a finite number of threads that can be dispatched due to the fact that resources are limited on the machine and at some point as the thread count becomes too high the cost of context switching begins to negatively affect performance as well. If more requests come in than there are threads to process those requests, the users will not be able to get to the site and will quickly get frustrated, reducing the overall sales numbers. The problem is that some requests take longer to process because they just do more stuff, and the stuff that I’m talking about here specifically are non-CPU bound tasks. Consider that you’re shopping at your favorite online store and you are in the process of checking out. After you fill in all of the pertinent information and hit the purchase button you have to incur at least one database call, a service call for payment processing and maybe the sending of a confirmation email of the order all before the response is sent to the client. In a traditional ASP.NET MVC website, this can look something like the following:
public class CheckoutController : Controller
{
private readonly IPaymentProcessor _paymentProcessor;
private readonly IOrderRepository _orderRepository;
private readonly IEmailGenerator _emailGenerator;
public CheckoutController(IPaymentProcessor paymentProcessor, IOrderRepository orderRepository, IEmailGenerator emailGenerator)
{
_paymentProcessor = paymentProcessor;
_orderRepository = orderRepository;
_emailGenerator = emailGenerator;
}
public ActionResult ProcessPurchase(OrderViewModel orderData)
{
orderData.ValidateData();
ProcessPaymentAndSave(orderData); //make various service calls (this can take some time)
return View("Confirmation", orderData);
}
private void ProcessPaymentAndSave(OrderViewModel orderData)
{
//collect the payment information
orderData.PaymentDetails.AuthorizationCode = _paymentProcessor.ProcessPayment(orderData.PaymentDetails);
//save the order to the database
orderData.OrderId = _orderRepository.Save(orderData);
//generate and send a confirmation email.
_emailGenerator.SendConfirmationEmail(orderData);
}
}
These types of instructions are I/O bound and can take a fair amount of time to complete, and the thread that is processing these requests has to wait for all of them to finish before being able to process another request.
Defending against the Horde
As I mentioned above, asynchronous patterns in .NET have been around for quite a while and they center around the idea that calls can be made asynchronously by breaking them into two parts, the beginning (call) part, and the end (callback) portion. Using this model the webserver still dispatches a thread to process the incoming request, but as the asynchronous call is made, the thread is returned to the pool to do process other incoming requests. Once the call has been completed ASP.NET is notified and the callback is queued onto the threadpool (most likely a different thread than the call was sent on, I might add) to pick up where it left off. This allows the webserver to be more efficient with thread management and more requests can be handled simultaneously as a result. It must be noted that to get the benefits of true asynchronous calls, it’s not sufficient to simply make the entry point asynchronous but rather to refactor the underlying long running I/O operations to also use their asynchronous counterparts. As of .NET 4.5, many classes in the framework have been augmented to include asynchronous methods that return Task or Task objects as opposed to the earlier asynchronous model of calling into the Begin/End methods provided on those classes. This means that if we are sending info to a payment gateway using HttpWebRequest, that we use the GetResponseAsync method rather than the synchronous GetResponse or the BeginGetResponse and EndGetResponse combination. In MVC3 and MVC4, there has been a convention for enabling asynchronous behavior, but it involves some significant refactoring and can be detrimental to readability and the flow of what’s actually happening because of all of the separation. Here’s our checkout process example written out in MVC4:
public class CheckoutController : AsyncController
{
private readonly IPaymentProcessorAsync _paymentProcessor;
private readonly IOrderRepositoryAsync _orderRepository;
private readonly IEmailGeneratorAsync _emailGenerator;
public CheckoutController(IPaymentProcessorAsync paymentProcessor, IOrderRepositoryAsync orderRepository, IEmailGeneratorAsync emailGenerator)
{
_paymentProcessor = paymentProcessor;
_orderRepository = orderRepository;
_emailGenerator = emailGenerator;
}
public void ProcessPurchaseAsync(OrderViewModel orderData)
{
orderData.ValidateData();
ProcessPaymentAndSaveAsync(orderData); //make various service calls (this can take some time)
}
public ActionResult ProcessPurchaseCompleted(OrderViewModel orderData)
{
return View("Confirmation", orderData);
}
private void ProcessPaymentAndSaveAsync(OrderViewModel orderData)
{
AsyncManager.OutstandingOperations.Increment(3);
//collect the payment information
_paymentProcessor.ProcessPaymentAsync(orderData.PaymentDetails)
.ContinueWith(paymentCollectionResult =>
{
orderData.PaymentDetails.AuthorizationCode = paymentCollectionResult.Result;
AsyncManager.OutstandingOperations.Decrement();
//save the order to the database
_orderRepository.SaveAsync(orderData)
.ContinueWith(saveResult =>
{
orderData.OrderId = saveResult.Result;
AsyncManager.OutstandingOperations.Decrement();
//generate and send a confirmation email.
_emailGenerator.SendConfirmationEmailAsync(orderData)
.ContinueWith(emailResult =>
{
AsyncManager.OutstandingOperations.Decrement();
});
});
AsyncManager.Parameters["orderData"] = orderData;
});
}
}
internal class SampleOrderRepository : IOrderRepositoryAsync
{
public Task<object> SaveAsync(OrderViewModel orderData)
{
var connectionString = ConfigurationManager.ConnectionStrings["orderDatabase"].ConnectionString;
using(var connection = new SqlConnection(connectionString))
using(var command = new SqlCommand("spInsertOrderData", connection))
{
connection.Open();
//... building parameters from the orderData not shown here for simplicity
return command.ExecuteScalarAsync();
}
}
}
As you can see the code is very different from its original form. MVC has a special controller type to enable asynchronous processing and each action that will be called asynchronously has to be created with Async and Completed conventions so that MVC knows how to handle the flow of the asynchronous call. Also, the AsyncManager class is used to track the outputs of the long running portions of code so that they can be passed into the callback once all of the data is aggregated. Also, we’ve converted all of our service interfaces to use asynchronous versions of their calls. An example of that is shown in the SaveAsync method of the SampleOrderRepository class. Instead of using the ExecuteScalar method, we call ExecuteScalarAsync, which returns a Task<object> instead of just object. The Task class serves as a representation of an asynchronous operation, but it also contains information about the operation taking place, like the current status of the operation and the result of the operation. The class also provides a set of continuation methods that allow for the specification of logic that should happen when the operation completes. The Task class sits on top of the Task Parallel Library and makes use of the new threadpool implementation from .NET 4.0 to efficiently schedule when the work takes place.
With the advent of the async and await keywords in combination with the Task class, the simplified syntax lets the compiler know that you want to create an asynchronous call with a callback method, and it can do the heavy lifting on its own which makes your code significantly simpler and easier to read and understand. Let’s take a look at how to accomplish the same asynchronous checkout process using the async/await keywords:
public class CheckoutController : Controller
{
private readonly IPaymentProcessorAsync _paymentProcessor;
private readonly IOrderRepositoryAsync _orderRepository;
private readonly IEmailGeneratorAsync _emailGenerator;
public CheckoutController(IPaymentProcessorAsync paymentProcessor, IOrderRepositoryAsync orderRepository, IEmailGeneratorAsync emailGenerator)
{
_paymentProcessor = paymentProcessor;
_orderRepository = orderRepository;
_emailGenerator = emailGenerator;
}
public async Task ProcessPurchase(OrderViewModel orderData)
{
orderData.ValidateData();
await ProcessPaymentAndSaveAsync(orderData); //make various service calls (this can take some time)
return View("Confirmation", orderData);
}
private async Task ProcessPaymentAndSaveAsync(OrderViewModel orderData)
{
//collect the payment information
orderData.PaymentDetails.AuthorizationCode = await _paymentProcessor.ProcessPaymentAsync(orderData.PaymentDetails);
//save the order to the database
orderData.OrderId = await _orderRepository.SaveAsync(orderData);
//generate and send a confirmation email.
await _emailGenerator.SendConfirmationEmailAsync(orderData);
}
}
internal class SampleOrderRepository : IOrderRepositoryAsync
{
public Task<object> SaveAsync(OrderViewModel orderData)
{
var connectionString = ConfigurationManager.ConnectionStrings["orderDatabase"].ConnectionString;
using(var connection = new SqlConnection(connectionString))
using(var command = new SqlCommand("spInsertOrderData", connection))
{
connection.Open();
//... building parameters from the orderData not shown here for simplicity
return command.ExecuteScalarAsync();
}
}
}
The first thing to notice here is that our MVC code doesn’t look so fragmented anymore. It’s quite representative of the synchronous model. As before, we’re using our asynchronous implementations of the services that perform our long running processes. There are three additional callouts here that all work together to make the call happen smoothly. The first is that the Action method now returns Task
That’s A Wrap
The Taskmatics Scheduler makes judicious use of the new async and await keywords in the administration website. In order to ensure that the UI is responsive when retrieving a lot of data, we make data retrieval calls asynchronously so that we can retrieve more data concurrently and therefore display the screen to the users as fast as possible. For us, using async and await translates into a better user experience when administering tasks through the website, and if you’re a developer for a major retailer’s online store this year, it could mean enjoying more delicious thanksgiving leftovers knowing that your site has an upper hand against the throngs of shoppers looking to cash in on the savings.