Python and The Very Slow Server

, in Computing

I don't usually do a lot of Python programming, but I always enjoy it when the opportunity arises . Python is in no way a "clean" language, it has all sorts of warts and limitations that mean that it tends to not get used for big projects. Despite this (or maybe because of it), Python remains my go-to language for Getting Small Things Done Quickly. It is impossible to overstate the utility of just being able to start coding a function by bashing away at the python console - nothing else has given me the same sense of instant gratification since I started programming in BASIC back in the 80s.

The other big advantage of Python is the useful utility libraries that come with it as standard. Want to send twenty thousand emails? Just import smtplib. Want to generate code based on data from a spreadsheet? Import csv and away you go. Need a file that is exactly 32Mb is size? No problem. These are real examples from my job where Python has saved me many hours.

The most recent use I have put Python to is a slow server. For various murky and uninteresting reasons I need a rate-limiting HTTP server, one that I can easily control the speed at which it sends data. Enter Python's very handy BaseHTTPServer module, which allows you to create custom HTTP servers with only a few lines of code by subclassing a request handler. Although the BaseHTTPServer is fairly useless for serving real files, it is perfect for this type of thing since it does all the boring work of parsing headers and returning status codes.

I don't care about the contents of the data, just its size and how long it takes to serve. Since I will be varying these parameters a lot, I decided to make them part of each request so that each request could take a different amount of time - this means I don't have to restart the server between each test run. Modifying the code to serve actual file data would be very simple.

I enjoyed writing this server so much that I regret that it didn't take longer. Now I actually have to use it for its intended purpose, which I can assure you is not going to be as pleasant.

Here is the complete Python source:

# A very simple HTTP server designed to for testing situations where the data returned
# is not important but the rate at which it comes down is. This server can be started
# using the command: python delayserver.py
#
# Once started, it will listen for requests on port 8000
# Requests should be of the form http://<address>:8000/size=<bytes>,duration=<seconds>
# where: <bytes> is the size of the response data
# and    <seconds> is how long you want it to take (at minimum, it may take longer)
#
# Notes:
# * The timing is pretty inaccurate for small byte sizes, this isn't a problem for
#   what I need it for
# * Press ctrl-c to stop serving


import time
import BaseHTTPServer

class MyHTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):

    def do_GET(self):
        request = self.path.strip("/")
        duration = 1
        size = 1024

      validRequest = False
        params = request.split(",")
        for p in params:
        temp = p.partition("=")
        if (temp[0] == "size"):
            size = int(temp[2])
            validRequest = True
        elif (temp[0] == "duration"):
            duration = int(temp[2])
            validRequest = True

        if (validRequest == False):
           self.send_error(404)
           return

        self.send_response( 200 )
        self.send_header( "Content-Length", str(size) )
        self.send_header( "Pragma", "no-cache" )
        self.end_headers()
        self.slowWrite( self.wfile, size, duration )


    def slowWrite(self, output, size, duration):
            bytesWritten = 0
            startTime = time.time()
            while ( bytesWritten < size ):
                    now = time.time()
                    if (duration != 0):
                            desiredBytes = ( (now - startTime) / duration ) * size
                    else:
                            desiredBytes = size
                    desiredBytes = min( size, desiredBytes )
                    if (desiredBytes < bytesWritten ):
                            time.sleep(0.2)
                    else:
                            while (bytesWritten < desiredBytes):
                                    output.write('A')
                                    bytesWritten = bytesWritten + 1
                            output.flush()
            now = time.time()
            self.log_message( "Request took %f seconds",   now - startTime  )

if __name__ == "__main__":
    http = BaseHTTPServer.HTTPServer( ('', 8000), MyHTTPRequestHandler )
    print "Listening on 8000 - press ctrl-c to stop"
    http.serve_forever()

I should point out that I am by no means an expert at Python, so take this code with a pinch of salt.