Calling Analytical Guru's: Challenge - Getting fastest web services...

[Administrator]

Calling all analytical .NET gurus that like to get the most out of .NET performance!

I've been doing some tests to see what the fastest way is to get a "dataset" of data via web services. Take an example DataSet containing 1,000 rows (or more if you want) then try every way you can think of to transfer it from your server to your client. Use the StopWatch class to measure the elapsed milliseconds in your test (ms is sufficient) and see which delivery method gives you the best speed.

I've been experimenting with retrieving plain strongly typed datasets, byte arrays, compressed byte arrays (DeflateStream class), then also throwing in some encryption and compression again and seeing what that brought.

My initial tests with smaller datasets concluded that the strongly typed dataset was fastest! Faster even than a compressed byte array. Granted, all web services are using my custom IHTTPHandler also employing gzip/deflate compression on every call, so the DataSet effectively is getting HTTP compression to help it along. I haven't tried larger datasets yet.

So for those interested, let's see if we can determine and/or come up with a way to deliver the FASTEST datasets from .NET 2.0.

No restrictions, see what you come up with!

 

[JohnH]

I've done pretty much the same tests as you, but in a limited test environment (local machine) - the observations still apply. Test data was 10MB Xml data with thousands of 'records' read into a Dataset.

I did one webmethod that returns a Dataset and one that returned a GZip compressed byte array of the Dataset. I combined tests with Http compression by IHttpHandler and by IHttpModule. (Linked here is my resources for these interfaces, both in C#.) I measured with StopWatch class and analyzed with Fiddler.

Not sure if I misunderstood, but IHttpHandler didn't get me anything, it only compress headers and not content? IHttpModule on the other hand clearly compressed the Http response content by a 10:1 ratio (down from 10MB to about 1.2MB). Combining compressed byte array return with IHttpModule compression actually increased the total response size again by 25% (to about 1.5MB), this also adds more processing time both for in class service compression and client decompression.

Webmethods were buffered by default setting, it's the same as setting the BufferResponse property True, this means the entire response is buffered before sending it to client (efficient and fast as they say).

With almost no bandwidth limit in test environment my best was plain DataSet combined with IhttpModule compression. This means that Http response compressing/decompression time is very fast. Looking at realistic use through internet there would be no compare of transferring 1MB and 10MB, even at fast connection.

I have to research more if to find optimizations, but I think there is interesting information on these alternatives here.

Attached is the VB.Net translation of the IHttpModule resource used.

 

[Administrator]

Thanks, great info! Actually I too am using IHTTPModule for compression, I mispoke. I would really like to see an example of using Binary Serialization passing DataSets this way and test the numbers as compared to what we have already tested. I tried putting the DataSets in a Class Library but I was unable to cast them for some reason! I may reattempt the binary serialization again, I just cannot believe how much time I've put into this!!! You "should" be able to just set the DataSet to binary serialization IMHO, it would then transport as a byte array, and be that simple! It's not due to this dynamic compilation model in ASP.NET 2.0.

 

[vis781]

You might want to have a look at the ISerializationSurrogate interface. With it you can override the default serialization of a dataset to xml.

Info can be found below......

http://msdn.microsoft.com/msdnmag/issues/04/10/CuttingEdge/

 

[Administrator]

I read this article and a few others referenced from it and googled on the ISS interface. I'm not sure I'm fully understanding what's going on here, but also what I don't want to have to do is have something that may be unique to one strongly typed dataset. I'd like to have one binary serializer/deserializer for all web services that pass through this serializer.

 

[Administrator]

Well, I'm happy to report success in using the binary formatter! Yes, I had to place my DataSets into a class library, and then add a property to the dataset designer so I could pass in the connection string as I don't want my connection strings publicly viewable in a app.config file. This normally isn't a problem as the connection strings are server side, but it is something I had to do to get things to work, as you'll see when you try to cast the strongly typed data.

Regardless of the nonsense above, this has been way more time than I had alotted for on this project, but hopefully in the end it will pay off in performant web services. After I got the binary ser/deserialization working I then added compression (deflate) and drastically reduced the byte array size being transported even further. I know there will be a small penalty to compress the bytes out and decompress on the client side, but this should pay off for very large amounts of data.

I don't having timing numbers, the articles vis781 provided shows that the straight up DataSet is probably better for small use, the binary format will hold its own when transporting bulks of data.

Well, time to refactor all of my web services now! There is another benefit of the class library approach, security! People cannot see your dataset architecture if you are returning the byte arrays vice returning the dataset!

 

[JohnH]

I'm not finished with this vis781 tip about the binary serialization is good, the best for performance, but it have to be combined with some compression. I didn't have to use surrogate, followed the Ado.Net 2.0 tip about RemotingFormat. Compressing with the BZip2 of SharpZipLib reduzed my initial package from 10MB to 502KB! For this I turned off IHttpModule. The Bzip2 is rather slow, but it have to be accounted for the data size and internet connection speed.

 

[Administrator]

One of my thoughts is/was is there going to be much difference in a compressed binary format vice a xml format? If I'm using compression, are they both going to end up the same? I believe the binary would actually be smaller due to the reduce amount of data in the binary format, vice the "wordy" XML serialization format. I'm going to keep the IHTTPModule on as I think that is something that really needs to be in place. There may be other calls that return values, such as smaller Datasets, in my scenario, that I may not want to put through this serialize, compress, retrieve, decompress, deserialize routine. Not sure yet! But my ultimate goal with the smart client era is to get data fast and minimize this delay. Background threads are great, paging and smart data pulls (not querying for everything!), but we still have to wait for the transport across the wire. Good tips in those articles too, just like we practice minimizing round trips to our data server, we also need to minimize round trips to the web service server. Returning DataSet arrays, or filling one DataSet will all the required return cargo, Structures which contain differing DataSets, are all thoughts in optimizing performance! Architect it right the first time, reuse later! Hence my new base classes for my company.

 

[Administrator]

I'm not finished with this vis781 tip about the binary serialization is good, the best for performance, but it have to be combined with some compression. I didn't have to use surrogate, followed the Ado.Net 2.0 tip about RemotingFormat. Compressing with the BZip2 of SharpZipLib reduzed my initial package from 10MB to 502KB! For this I turned off IHttpModule. The Bzip2 is rather slow, but it have to be accounted for the data size and internet connection speed.

John,

What file size do you get when you use the DeflateStream instead of BZip? I've actually been pretty happy with the .NET 2.0 Compression class.

 

[JohnH]

Time for some stats, attached as image. I will try to make this short, having done a lot of different tests here. (Short answer Neal, I would probably go for Bzip2 and Binary serialization in a plain http webservice under these circumstances. Returning a Dataset and using only Http compression is a good performer, too.)

Test set
My Xml input file is 10MB, don't know how much this is in-memory Dataset, probably less due to field/schema layout vs Xml tags. I locally tested transfer of Dataset or Bytes array. For bytes array I tested 3 different conversion methods; (1)WriteXml/ReadXml, (2)Binary serialization and (3)Xml serialization, and these were combined with internal compression types (a)none (b)Gzip (c)Deflate and (d)Bzip2. All tests were combined with/without Http compression through IHttpModule, different compression types can be done for Http compression also, I used only Deflate here throughout for speed and simplicity of tests.

Stats
Measured size of Dataset byte arrays, conversion time (to+from), Http transfer bytes (data always inflate when wrapped in a soap message and transferred with the http protocol).

Interpreting
Gzip and Deflate is pretty equal, Bzip2 compress to one third of these, but is a few seconds slower. Generally it is best to leave out Http compression if data is already compressed. Best times and less bytes is bolded in the stats. Binary serialization is always fastest. For the .Net compression methods that both are Deflate-based the Xml methods make less bytes. Remember that use of the web service through internet is limited to connection bandwidth (both parties) and server CPU time, and 5 seconds extra processing time could easily beat 1MB extra transfer in overall competition. The actual content of the Dataset will impact the performance, different type content behave differently in compression environments.

 

[Administrator]

Great info!!! So to me, from your chart, the best of all of them is deflate + BinaryFormatter. One other technique of course to keeping the most performant web services is to also enable the cachine. CacheDuration:=60 for example in the WebMethod attributes. Just like output caching asp.net pages/usercontrols.

Do you know how to modify the IHTTPModule to not compress Byte Arrays going out compressed? It would be nice, per your suggestion not to compress already compressed streams. I'm not sure if it is going to save much time, but I guess a header check or something of the outgoing stream could be done to skip further compression? For me "caveman approach" I'm thinking of just leaving the IHTTPModule as is. However, on the same note, how can we configure it to handle exclusions? For example, what if I don't want something to be compressed, i.e. I've had problems writing PDF's to the response stream before and had to bypass the ICSharpCompress system. I'll probably take a look at their code as I know you could append cx=off to the URL and compression is skipped. Hmmmm, lot's to think about...

Great work! Going to be interesting to see how this new smart client performs using .NET 2.0 features.

 

[Administrator]

Here is what I added to my IHTTPModule so you can bypass compression by passing ?cx=off to the URL:

Public Sub OnBeginRequest(ByVal sender As Object, ByVal e As EventArgs)
Dim app As HttpApplication = sender
'bypass compression if cx=off

Dim param As String = app.Request.Params("cx")
If param IsNot Nothing AndAlso param.ToLower = "off" Then

Return

End If

If IsEncodingAccepted("gzip") Then

app.Response.Filter = New GZipStream(app.Response.Filter, CompressionMode.Compress)
SetEncoding("gzip")
ElseIf IsEncodingAccepted("deflate") Then

app.Response.Filter = New DeflateStream(app.Response.Filter, CompressionMode.Compress)
SetEncoding("deflate")
End If

End Sub

 

[JohnH]

That is not working here, and I don't see how it could. It is not the request that determines if application method is compressing its content internally, the request only allow accepted Http encodings. Also, the response.Filter is applied before the requested method is invoked/processed, not knowing what would be returned.

What can be done with a web service that serves many methods is to rule out Http compression according to what method is requested, you can do this by checking the HttpApplication.Request.PathInfo in BeginRequest event. When a method "myMethod" is processed PathInfo property value will be "/myMethod". This requires hardcoding those method calls you don't want default Http compression for. Example

If app.Request.PathInfo = "/getDScompressed" Then Return
 
'...else allow compression for accepted Http encodings...

 

[JohnH]

aha, just got your point, you manually add the string "?cx=off" to the URL in the client application.. well, that could be an option if you hardcode url requests - can this be done when invoking web service methods from service instances in code, too?

Anyway, my suggestion is easier to maintain at service point

 

[Administrator]

Hmmm, thanks! I haven't tested my method, I guess no need to now! I would like to have some type of flag I can pass in that the IHTTPModule could determine whether or not to bypass compression. I'll explore your idea more, thanks!