Best Way To Send Multiple Images Sequentually Over a Network Stream

bigfoot3d

Well-known member
Joined
Mar 11, 2007
Messages
56
Location
New Orleans, LA
Programming Experience
3-5
I am sending images from a client computer to a console computer. The client computer sends the image to a server, and then the server turns around and forwards it to the console.

In 1280 x 1024, the jpeg image size is 126kb. Currently my server only does 158kbs down (0.94 seconds) and 70kbs (2.13 seconds) up, which is typical going through a home cable internet connection.

Is there anyway to compress or reduce the size of the image without reducing the resolution. I didn't know if the best thing was to compress on the client side through a memory stream before sending it to the network stream and then uncompressing on the console side after recieving the network stream using a memory stream. Or should i just compress to the network stream on the client side and uncompress from the network stream on the console side . Anything lower than 1280 x 1024 is not easily readable on the console side.

I know some would say just up the speed of the connection to the server, but my problem is am also trying to factor in the speed of connecting to homeusers computers as well, so i can't really eliminate the speed factor. I know there is someway to do this because in order to make remote desktop work properly and smoothly some kind of compression must be going on in order to smoothly transmit the remote desktop connection to my server from the internet.

Any insight on this would be greatly appreciated!

Thanks,
Craig:confused:
 
Jpegs are already compressed, so you can work with the compression ratio of the jpeg encoding to find best size/quality that works in this case. Other than that there is the "framerate", how often the image is sent. There shouldn't be many bytes to benefit from zipping it also, plus that adds CPU time at both ends.
See this about Jpeg compression http://support.microsoft.com/?kbid=324788 (I think the default is around 70%)
1280 resolution does sound a bit on the high end, 800 res would be more normal for remote desktoping.
 
You wouldnt believe the number of tricks pulled to make remote desktop work in a responsive manner. It takes a team of brilliant people to think of and maintain it (UltraVNC.org for example, or the team of people responsible for Terminal Server at Microsoft) so do be careful that you arent biting off more than you can chew here.

Without meaning to sound condescending, I think this task is a little too far beyond your programming capability/experience/imagination right now, if only immediately for the fact that a sane person would NEVER use JPEG as a compression device for screenshots. JPEG is intended for photos, where gradual changes of contrast/colour over a diagonal scan of an 8x8 pixel block tile experienced. Harsh changes of contrast/colour are totally screwed up by jpeg and suffer heavily from blocking and ringing, as can be seen by anyone who ever posted a screenshot saved as a JPEG.
 
Did you say Png? Or perhaps Emf? Actually I'm the only to mention Jpeg here. For a plain screenshot I get a few KB less size and much (!) improved quality with Png/Emf, but I have seen strange things happening with Png's in some situations/apps where the size suddenly becomes up to 10 times larger than Jpg. I haven't figured out why that happens sometimes. In addition there was problem with transparency of these in IE6- so I avoided and forgot it. You should check these formats bigfoot3d.
 
Image Formats and Sizes

First of all thank you both for your input.

I had already done the testing with the different image formats, and these are my results @ 1280 x 1024

101KB - GIF
126KB - JPEG
305KB - EMF
305KB - PNG
305KB - WMF
305KB - EXIF
305KB - ICON
305KB - MemoryBMP
631KB - TIF

I am not going to give up on this that easily. I started on this 2 months ago, and I can already chat with the client PC, and control the clients mouse and do mouse clicks, and even type words etc. on the clients PC. There is no problem with this working on a LAN, but i want to now move it to over a WAN, so that is why i am exploring the graphic format and size options. This is the only really thing holding me back now is the size of the image. I am going to look more into JohnH first post on this topic. I am using this to record the chat session, client info at connect, and amount of time it takes to fix the problem.

I'll keep hammering this out. Any other thoughts are appreciated as well :)
 
Using Compression

WOW, I am getting there. JohnH, thanks once again for your input. The site you gave in post 2 helped me out. I copied and pasted the following code:

VB.NET:
    Private Sub SaveJPGWithCompressionSetting(ByVal image As Image, ByVal szFileName As String, ByVal lCompression As Long)
        Dim eps As EncoderParameters = New EncoderParameters(1)
        eps.Param(0) = New EncoderParameter(Encoder.Quality, lCompression)
        Dim ici As ImageCodecInfo = GetEncoderInfo("image/jpeg")
        image.Save(szFileName, ici, eps)
    End Sub

0 being the highest and 100 being the lowest amount of compression.

I did a compression of 0 first and the file came out to be 29KB. However it was blurred.

Then I switched to a compression of 25 and the file came out to be 64KB and was perfectly readable.

Then I switched to a compression of 15 and the file came otu to be 52KB and was ok, but not something I would want to work with.

For the heck of it, I tried a compression of 20 and the file came out to be 60KB and was a little better.

So as of right now I will go with the compression of 25 and see if there is anything else I can do to simplify the size, but retain quality.

Once again, any other thoughts would be greatly appreciated!
 
Can you use the Remote Desktop feature already built into windows? Or VNC?

These boys essentially only bother transmitting parts of the image that have changed, or the foreground window, or they transmit drawing commands only (Xwindows) - very cool, high tech, low level tricks. Are you sure you want to reinvent their wheel?
 
Basically Done - Just Improving

I looked into Remote Desktop and VNC before I started. 1 if someone was behind a firewall, and needed help, they would run the app on their pc which would come out to a server, which bypasses me having to worry about getting through their firewall from the outside, and 2 what I wanted was something that gives me complete control and complete logging capabilities, as I will be tying this into a much large database that I will be creating. Client and tech logins will be recorded as well as amount of time lapsed to solve the problem.

I have pretty much reinvented it. As I said before I can completely control the clients PC. I can see the clients PC as well and have been testing out installing, troubleshooting, etc all by controlling the clients PC's keyboard and mouse function from my PC's keyboard and mouse using my program. I can also perfectly view the client PC. I was just looking for a better way to view the clients PC across the internet where bandwidth become an issue. I can work the software without a glitch on a Local Area Network. This is all tying into a bigger build.

Bottom line is that I am for the most part done. I am just playing with the imaging side of it to see if there is anyway I can maintain quality but somehow reduce memory size required for the image itself.

So basically I have spent 2 months reinventing the wheel. I knew that before I went into it. I am satisfied with where I am at right now. I have a completely functioning program when it come to a LAN Connection.

I am just looking to make some improvements for view the client desktop using the software through an Internet Connection.

Once again, any other thoughts or comments would be greatly appreciated!
 
I believe some of the VNC implementations are open source, so you can see what tricks they pull. SPeaking to the developers might also provide some valuable insight..
 
Good Idea!

Cjard,

Thank you for the input! I will definately look into some open source and possibly speaking to some of the developers if I cannot figure out what is going on in the sourcecode. So that is the way I will plunder onto now.

Thanks again!

If anyone else wants to throw and opion in, feel free! :)
 
After moving on to other things and getting them to work, I am now back on this project to complete it. However I don't believe the way I was looking at it before was the correct way to go about sending the images across a network stream. I believe I am now on the right track with using the Graphics/GDI+ tricks to make this work.

This is to send images (in this case screenshots) sequentially over a network stream.

I am just writing this post in case anyone else is trying to do the same thing I was doing and trying to point them in the right direction!

This new trick I am trying to accomplish is posted is in the Graphics/GDI+ Forum.

http://www.vbdotnetforums.com/showthread.php?t=22712
 
Not related to your project, but my company uses WebEx to get around the firewall problem for remote support. They market it as a remote meeting product, but you can watch and control other people's desktops very easily. It has been a lifesaver.
 
Yeah, pretty much the same as Log Me In. I have already designed the complete managing software for my company this past year. Now I am just on the move to design a remote application to help out customers that will tie directly in to what I have already started.

Thanks for the idea though :p
 
Here's an idea, use LockBits method of Bitmap to access the image bytes directly. With a quick integer comparison you can extract only the pixels that have changed since last screen, to do this you should operate with 32bpp images where each pixel is 4 bytes, ie Integer. This will give far less memory read operations and comparisons than with a 24bpp. I have found that encoding the image representing the changes to Png and then compressing this with Deflate give the lowest number of bytes to be sent over the network.

Example of LockBits method follows, I'll just explain three variables not defined in this code, oldscreen and newscreen is the 32bpp images to compare, rct is a Rectangle representing the Bounds of the images and same as Screen.PrimaryScreen.Bounds for a full screenshot. Pixels that are identical is reset to 0 in newscreen, so make a copy (clone) of this beforehand because "newscreen" will be the "oldscreen" for next time.
VB.NET:
Dim bmdata1 As Imaging.BitmapData = oldscreen.LockBits(rct, Imaging.ImageLockMode.ReadOnly, Imaging.PixelFormat.Format32bppArgb)
Dim bmdata2 As Imaging.BitmapData = newscreen.LockBits(rct, Imaging.ImageLockMode.ReadWrite, Imaging.PixelFormat.Format32bppArgb)
For x As Integer = 0 To rct.Width - 1
    For y As Integer = 0 To rct.Height - 1
        Dim pos As Integer = (x * 4) + (y * bmdata1.Stride)
        Dim px1 As Integer = Runtime.InteropServices.Marshal.ReadInt32(bmdata1.Scan0, pos)
        Dim px2 As Integer = Runtime.InteropServices.Marshal.ReadInt32(bmdata2.Scan0, pos)
        If px1 = px2 Then
            Runtime.InteropServices.Marshal.WriteInt32(bmdata2.Scan0, pos, 0)
        End If
    Next
Next
oldscreen.UnlockBits(bmdata1)
newscreen.UnlockBits(bmdata2) 'this newscreen now only contain the changes
This method is also quite fast, 0.3 seconds to compare all pixels and get image of changes on my at the time dated computer.

To encode Png and compress:
VB.NET:
Dim mem As New IO.MemoryStream
newscreen.Save(mem, Imaging.ImageFormat.Png)
Dim pngbytes() As Byte = mem.ToArray

mem = New IO.MemoryStream
Dim comp As New System.IO.Compression.DeflateStream(mem, IO.Compression.CompressionMode.Compress)
comp.Write(pngbytes, 0, pngbytes.Length)
comp.Close()
Dim compbytes() As Byte = mem.ToArray
'compbytes is "ready to ship"
Depending on the change the number of bytes to transmit could be significantly lower compared to a full detail image.

At receiver you must decompress and open the bitmap from the pngbytes. Then merge the received image of changes with previous image. The merge process is same as the extraction but instead of comparing px1 = px2 you check if px2 <> 0, this is a pixel that is changed from previous image. For example to write the changed pixel from newscreen into oldscreen:
VB.NET:
If px2 <> 0 Then
    Runtime.InteropServices.Marshal.WriteInt32(bmdata1.Scan0, pos, px2)
End If
To make this work both parties need an initial full screenshot, from there only changes need to be transmitted.
 
JohnH,

Thanks once again for your most valuable insight. This is definitely a nice alternative to getdibits and setdibits that will work.

If i ever get getdibits and setdibits to work, i will compare the speed of the processing against the lockbits.

The only thing I need to figure out now is how to convert an image to Imaging.PixelFormat.Format32bppArgb.

I am testing this with 2 computers right now, one with a 32 bit card, the other only supports 24 bit. I get the following failure:

AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

At the following line of code:
VB.NET:
 Dim px2 As Integer = Runtime.InteropServices.Marshal.ReadInt32(bmdata2.Scan0, pos)

I am suspecting this is because the sending machine is sending the image in 24 bit format or because the machine cannot handle the 32 bit and it is not being handled properly at that line of code because of format differences.

Either that or in order for this to work the machine being remoted to must have a 32 bit graphics card in order to support the program?

I also remoted into a computer and while I was remoting in, it was set to 16bit, however the graphics card on that pc still supports 32 bit so I am thinking this is why this worked on this machine.

If I run this against the same 32 bit computer using 2 screens, it works just fine, so I am suspecting that it is a 32 bit problem.

So bottom line, in order to make the 32 bit work the graphics card must support 32 bit and anything else will fail?

Just getting my facts straight. :p

Thanks!

Update: I installed a 32 bit graphics card on that machine and got the same result, maybe its just that machine acting up, I will have to test this out on a few more machines and see what happens.
 
Last edited:
Back
Top