Engineering Challenges Q&A - iseahound/ImagePut GitHub Wiki

Challenges encountered when developing ImagePut

General Questions

Q: Why does the following regex ([a])+ fail for long strings?

A: I don't know. AutoHotkey hates the 1+ operation and will only permit it for short input strings. As a workaround, use the Kleene star (possessive) ([a]{2})*+. instead. The counter bracket is irrelevant, ([a][a])+ would fail also. This workaround is used in the detection of hexadecimal and base64 strings.

Q: What is the shortest possible image encoded into base64 and hex?

A: They are:

; 24 bytes. A black pixel. Unstable.
ImagePutWindow("R0lGODlhAQABAAAAACwAAAAAAQABAAAC")
ImagePutWindow("474946383961010001000000002c00000000010001000002")

; 33 bytes. Smallest possible 1x1 transparent pixel.
ImagePutWindow({base64: "R0lGODlhAQABAAAAACH5BAEAAAAALAAAAAABAAEAAAIA"})
ImagePutWindow({hex: "4749463839610100010000000021f90401000000002c0000000001000100000200"})

For a nice 32 base64 and 48 hex string length.

Source: https://stackoverflow.com/questions/2570633/smallest-filesize-for-transparent-single-pixel-image

Q: What are the best practices for dealing with bitmaps and streams?

  • Always clone your stream as the seek position may be controlled by another program
  • Always clone your GDI+ bitmap as:
    • it may be corrupted
    • it may have negative stride

GDI

Q: Can the mysterious stock bitmap ever be leaked?

A: No. Although it cannot be retrieved via GetStockObject, it can be retrieved by the following code:

; Get a pointer to the mysterious stock bitmap.
; See: https://devblogs.microsoft.com/oldnewthing/20100416-00/?p=14313
MsgBox % obm := DllCall("CreateBitmap", "int", 0, "int", 0, "uint", 1, "uint", 1, "ptr", 0, "ptr")

GDI+

Q: What is the smallest image accepted by GDI+?

A: At least when using the CloneImage and CreateBitmapFromScan0 functions, the smallest valid bitmap is 1x1 or else it returns 0 as the pointer.

Q: What is an image and what is a bitmap?

A: Within the context of GDI+, an image is either a metafile or a bitmap. Under the Liskov substitution principle, all image functions work on bitmaps.

Q: Can a DC or a BITMAP be retrieved from a Graphics object?

A: Yes, but they will be write-only as a graphics object is write-only. Any drawn pixels cannot be retrieved.

For a special case where this is not true, I believe that if you already have access to the device context, then create a graphics object on top of it, and retrieve the device context from the graphics object, the two handles to the device context should be identical. This becomes a mechanism whereby passing the graphics object is equivalent to passing the device context itself. This can be done by calling the following functions in order: CreateCompatibleDC, CreateDIBSection (hbm), and SelectObject to put the hbm on the hdc. In other words, passing a pGraphics from the hdc or just the hdc (and creating a pGraphics later) is the same thing.

Q: How to clone a bitmap?

A: Call CloneImageArea on the bitmap to preserve the original image pixel format. Note that cloning creates a wrapper. See: https://stackoverflow.com/a/13935966

Q: Can I clone a bitmap clone?

A: Yes! In fact, clones of clones reference the original backing source such as a stream.

Q: Can the original bitmap be deleted before the clone?

A: Yes! This isn't C#, where there is a lock on the original bitmap.

ImagePut.gdiplusStartup()
DllCall("gdiplus\GdipCreateBitmapFromFile", "wstr", "cats.jpg", "ptr*", pBitmap:=0)
DllCall("gdiplus\GdipCloneImage", "ptr", pBitmap, "ptr*", pBitmapClone:=0)
DllCall("gdiplus\GdipDisposeImage", "ptr", pBitmap)
ImagePut.Show(pBitmapClone)

No! If you're using GdipCreateBitmapFromScan0, you have to maintain the integrity of the original buffer. The code below fails because the bitmap was disposed.

x := ImagePutBuffer("24.jpg")
DllCall("gdiplus\GdipCreateBitmapFromScan0"
, "int", x.width, "int", x.height, "int", 0, "int", 0x26200A, "ptr", x.ptr, "ptr*", &p:=0)
DllCall("gdiplus\GdipCloneImage", "ptr", p, "ptr*", &s:=0)
DllCall("gdiplus\GdipDisposeImage", "ptr", p)
ImageShow(s)

Q: Does cloning a bitmap change its pixel format?

A: No! It does not change whether using GdipCloneImage or GdipCloneBitmapAreaI. Moreover, it does not change 24bpp images to 32bpp images, nor does it act differently when preseneted with a bitmap of negative stride. Any notion of negative stride is always converted to positive stride when LockBits is called. Any pixel format conversions are always executed correctly when LockBits is called. This includes images that have a palette attached to them. So there is no need to directly specify a pixel format with GdipCloneBitmapAreaI unless you'd like to convert formats. Additionally, the backing streams are maintained with both cloning functions. Therefore, GIFs will still remain animated for example.

Q: Can image pixels be immediately loaded into a bitmap? (non-lazy operation)

A: Call GdipImageForceValidation immediately after bitmap creation. This operation fails silently if not called immediately following bitmap creation.

The following code will fail silently. All DllCalls return 0 indicating success. This outcome will not change even if sleeps are added between lines, or asynchronous non-blocking timers are used.

; Incorrect code.
pStream := ImagePutStream("https://picsum.photos/500")
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", pBitmap1:=0)
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", pBitmap2:=0)
DllCall("gdiplus\GdipImageForceValidation", "ptr", pBitmap1)
DllCall("gdiplus\GdipImageForceValidation", "ptr", pBitmap2)
ImagePut.Show(pBitmap1, "1") ; ImagePut.Show() does not clone the bitmap.
ImagePut.Show(pBitmap2, "2") ; This image is corrupted or blurred.

Correct. Validation must be collated to each creation of the bitmap.

pStream := ImagePutStream("https://picsum.photos/500")
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", pBitmap1:=0)
DllCall("gdiplus\GdipImageForceValidation", "ptr", pBitmap1)
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", pBitmap2:=0)
DllCall("gdiplus\GdipImageForceValidation", "ptr", pBitmap2)
ImagePut.Show(pBitmap1, "1") ; ImagePut.Show() does not clone the bitmap.
ImagePut.Show(pBitmap2, "2")

Note that the second bitmap is corrupted / blurred. This implies that the first bitmap has the superior lock (?). It is unknown why the first bitmap is not corrupted, as I expected. (The use of ImagePut.Show() over ImagePutShow() bypasses the use of CloneImage on the input bitmap.)

Q: How to convert between an HBITMAP and a BITMAP with transparency?

A: The solution is to use LockBits and a custom pointer to the image memory.

Hey, I'm glad you all enjoy this code, but I don't recognize it at the moment. For people like me who are confused, the problem is that if a hBitmap has transparent pixels, they will be converted to black using the built-in function.

Problem: Transparent pixels are 0x00000000. Alpha = 0x00, Red = 0x00, Green = 0x00, Blue = 0x00. In conversion, I assume the alpha channel is dropped. So the transparent pixel becomes 0x000000 which is black. Solution: We know a Bitmap can hold 32 bits of color. So, we should be able to preserve transparency.

Words:

  • ARGB = Alpha, Red, Green, Blue. Also known as 32-bit color.
  • pARGB = premultiplied ARGB. This is 24-bit data. The alpha channel has been multiplied into the color channels. Red = Alpha x Red. Green = Alpha x Green. Blue = Alpha * Blue.
  • Bitmap = pBitmap, pointer to a bitmap. This has two layers, one layer of struct data, and one layer of actual pixel colors. This is a flexible approach to holding information because the 1st layer can describe what information it is holding, and the second layer can be any data.
  • hBitmap = handle to a bitmap. This also has 2 layers, but the 1st layer contains limited information, only enough to convert a 1-dimensional array of pixels into a 2D image by storing the width, height, and some other small numbers. The reason for this is because an hBitmap is for displaying a graphical user interface, so your monitor only has red, green, and blue pixels. It does not have an "alpha" pixel. So, the pixel format of all hBitmaps is pARGB, and is 24-bit data.
  • pBits = pointer to the bits. This is the layer of actual pixel data.
  • DIB = Device independent bitmap = This is a hBitmap with the pixel data stored on memory. Memory = your computer's RAM. A long time ago, memory was expensive. So your computer monitor has a buffer (memory) that it uses to hold the image briefly before drawing onto the screen. Why not use your monitor's memory and save money? That's called a device dependent bitmap or DDB for short. Today, your printer still uses DDB. And you can access the memory buffer on your printer from your computer!
  1. We have a handle to our image, and we want to convert that handle to a pointer. The reason why we do this is because a pointer is global (can be transferred to more processes) while a handle is opaque, and probably has some limitations.
  2. Extract some information from our handle. Call GetObject and get width, height, and bpp. (bits per pixel)
  3. Now the most straightforward approach would be to find where the pixel values are in our hBitmap and memcpy the array of pixels to the pBitmap. This fails. Reasons #1 - The hBitmap could be bottom-up or top-down. #2 - hBitmap is pARGB (24-bit data) and ARGB is 32-bit data.
  4. We could just code everything in C, but we are going to avoid super low level stuff and be smart. We'll use some tricks. The first trick is to determine whether a bitmap is bottom-up or top-down. The standard approach is to change the value of the first pixel value in our data to a salmon-colored pink and call GetPixel. If we get a salmon-colored pink, then it's top down. If not it's bottom up. We won't do this. Instead, we'll create a new DIB (Device independent bitmap) which is the same thing as an hBitmap where the pixels are stored on local memory and copy the pixels using BitBlt. If our newly created DIB has is initialized with a negative height, then the pixels will be stored top-down. top-down is what a normal person should expect, like reading a book. Imagine reading a book from the last sentence on the page and making your way up. That's called bottom-up and it is how Windows normally stores pixel data. That's because some mathematicians were really concerned about purity and ideology. You know how a xy graph looks like? x-axis and y-axis? Yeah, the mathematicians wanted to use that model.
  5. For the second trick we'll use something special, a hidden flag in GdipBitmapLockBits that lets us determine where the return data will be buffered. You see, we get to choose where the output of the function will be, and I choose to put it where my DIB is. Importantly, I set another flag to display the output data as PARGB, the same format as a DIB.
  6. Do the following in order: Create a DIB. Create a Bitmap. Lock the Bitmap and point the output to the DIB. Use Trick #1 - copy the pixels to the DIB, erasing any top-down/bottom-up distinctions. Use Trick #2 - Unlock the bits, which copies the 24-bit pARGB data in a DIB into 32-bit ARGB pixels.
  7. Clean up and return the bitmap.

Q: Can a GDI+ Bitmap have negative stride?

A: Yes. The Scan0 must point to ptr + (Height-1)*Stride. Stride must also be negative in GdipCreateBitmapFromScan0.

Q: What does LockBits actually do?

A: (1) It copies the data into a buffer, which can either (a) allocated by GDI+ (b) allocated by the user. (2) It converts from one pixel format to the other. (3) It removes any negative stride.

Streams

Q: Is the fastest way to load a file SHCreateStreamOnFileEx?

A: No! In order of fastest to slowest:

  1. GlobalAlloc and CreateStreamOnHGlobal.
  2. SHCreateStreamOnFileEx
  3. GdipLoadImageFromFile
  4. GdipCreateBitmapFromFile

#4 was rather surprising!

from_file(filepath) {
    DllCall("shlwapi\SHCreateStreamOnFileEx"
                ,   "wstr", filepath
                ,   "uint", 0               ; STGM_READ | STGM_FAILIFTHERE
                ,   "uint", 0x80            ; FILE_ATTRIBUTE_NORMAL
                ,    "int", False           ; fCreate should be False to open the file.
                ,    "ptr", 0               ; pstmTemplate (reserved)
                ,   "ptr*", pFileStream:=0
                ,   "uint")
    DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pFileStream, "ptr*", pBitmap:=0)
    ObjRelease(pFileStream)
    return pBitmap
}

Q: What is the distinction between a Stream and a MemoryStream?

A: Streams are created by CreateStreamOnHGlobal. MemoryStreams are created by SHCreateMemStream. The main difference is that a stream supports GetHGlobalFromStream, retrieving a handle to its underlying memory, while a memory stream does not.

See: https://learn.microsoft.com/en-us/windows/win32/api/shlwapi/nf-shlwapi-shcreatememstream

Q: If a MemoryStream is cloned, can a handle to its memory be retrieved with GetHGlobalFromStream?

A: Surprisingly yes. Only works on Windows 8+ as previous versions did not support IStream::Clone.

; Create an IStream.
pStream := ImagePutStream("cats.jpg")

; Get the backing hGlobal from the stream.
DllCall("ole32\GetHGlobalFromStream", "ptr", pStream, "uint*", &handle:=0)

; Get a pointer and size to the encoded image.
ptr := DllCall("GlobalLock", "ptr", handle, "ptr")
size := DllCall("GlobalSize", "uint", handle)

pMemStream := DllCall("shlwapi\SHCreateMemStream", "ptr", ptr, "uint", size, "ptr")

; Fails to retrieve a handle to the memory as expected.
; DllCall("ole32\GetHGlobalFromStream", "ptr", pMemStream, "uint*", &handle:=0, "hresult")

; Succeeds???
pMemStreamClone := StreamToStream(pStream)
DllCall("ole32\GetHGlobalFromStream", "ptr", pMemStreamClone, "uint*", &handle:=0, "hresult")

StreamToStream(image) {
    ; Creates a new, separate stream. Necessary to separate reference counting through a clone.
    ComCall(Clone := 13, image, "ptr*", &pStream:=0)
    ; Ensures that a duplicated stream does not inherit the original seek position.
    DllCall("shlwapi\IStream_Reset", "ptr", pStream, "hresult")
    return pStream
}

Q: Does CreateStreamOverRandomAccessStream really return the same stream each time?

A: Yes. The user is responsible for cloning the stream to make it unique. CreateStreamOverRandomAccessStream should be renamed GetStreamFromRandomAccessStream.

#Requires AutoHotkey v2
p1 := ImagePutRandomAccessStream("https://picsum.photos/500")
p2 := get_RandomAccessStream(p1)
p3 := get_RandomAccessStream(p1)
MsgBox p2 "`n" p3, "Are the two pointers the same?"
ComCall(5, p2, "uint64", 1234, "uint", 1, "uint64*", &current:=0)
MsgBox (ComCall(5, p3, "uint64", 0, "uint", 1, "uint64*", &current:=0), current), "seek pointer"

get_RandomAccessStream(image) {
    ; Note that the returned stream shares a reference count with the original RandomAccessStream's internal stream.
    DllCall("ole32\CLSIDFromString", "wstr", "{0000000C-0000-0000-C000-000000000046}", "ptr", CLSID := Buffer(16), "hresult")
    DllCall("ShCore\CreateStreamOverRandomAccessStream", "ptr", image, "ptr", CLSID, "ptr*", &pStream:=0, "hresult")
    return pStream
}

Q: How does reference counting work with Streams and RandomAccessStreams?

A: A RandomAccessStream is always created over a Stream despite any attempt to imply otherwise. So, when creating a Stream from a RandomAccessStream, the internal stream is returned. As such, this internal stream always starts with a reference count of 2, 1 for the RandomAccessStream, and another for itself.

Note how the second stream has a reference count of +1 the first stream. They are the same stream!

p1 := ImagePutRandomAccessStream("https://picsum.photos/500")
MsgBox % "pRandomAccessStream References: " GetRefCount(p1) ; Displays 1.

p2 := get_RandomAccessStream(p1)
ObjAddRef(p2)
ObjAddRef(p2)
ObjAddRef(p2)
ObjAddRef(p2)
ObjAddRef(p2)
MsgBox % "First pStream References: " GetRefCount(p2) ; Displays 7.

p3 := get_RandomAccessStream(p1)
MsgBox % "Second pStream References: " GetRefCount(p3) ; Displays 8.

GetRefCount(p) {
    ObjAddRef(p)
    return ObjRelease(p)
}

get_RandomAccessStream(image) {
    DllCall("ole32\CLSIDFromString", "wstr", "{0000000C-0000-0000-C000-000000000046}", "ptr", &CLSID := VarSetCapacity(CLSID, 16), "uint")
    DllCall("ShCore\CreateStreamOverRandomAccessStream", "ptr", image, "ptr", &CLSID, "ptr*", pStream:=0, "uint")
    return pStream
}

Yet creating a RandomAccessStream over a Stream returns a new RandomAccessStream each time! Each RandomAccessStream starts with a reference count of 1.

p1 := ImagePutStream("https://picsum.photos/500")
MsgBox % "pStream References: " GetRefCount(p1) ; Displays 1.

p2 := set_RandomAccessStream(p1)
ObjAddRef(p2)
ObjAddRef(p2)
ObjAddRef(p2)
ObjAddRef(p2)
ObjAddRef(p2)
MsgBox % "First pRandomAccessStream References: " GetRefCount(p2) ; Displays 6.

p3 := set_RandomAccessStream(p1)
MsgBox % "Second pRandomAccessStream References: " GetRefCount(p3) ; Displays 1.

GetRefCount(p) {
    ObjAddRef(p)
    return ObjRelease(p)
}

set_RandomAccessStream(pStream) {
    DllCall("ole32\CLSIDFromString", "wstr", "{905A0FE1-BC53-11DF-8C49-001E4FC686DA}", "ptr", &CLSID := VarSetCapacity(CLSID, 16), "uint")
    DllCall("ShCore\CreateRandomAccessStreamOverStream"
                ,    "ptr", pStream
                ,   "uint", BSOS_PREFERDESTINATIONSTREAM := 1
                ,    "ptr", &CLSID
                ,   "ptr*", pRandomAccessStream:=0
                ,   "uint")
    return pRandomAccessStream
}

Q: What does GdipCreateBitmapFromStream actually do?

A: Yes. In fact it does several things.

  1. Adds 3 references to the stream.
  2. Sets the seek pointer to 4096. If the bitmap is read, the seek pointer will point at the end of the file.
  3. Should the seek position be reset to the beginning, all reads from the bitmap will be completely blank. However, this does not affect generic stream read/writes.
  4. Requires a call to GdipValidateImage to fully read the image and "free" the stream if you intend to utilize the underlying stream for something else. The seek position is then set to the end of the file.
  5. Should never create more than one bitmap based on a single stream. Otherwise, only part of the stream will be loaded when all bitmaps are queried at once. Most likely a data race.
  6. If you happen to have more than one bitmap on a single underlying stream, you can call GdipCloneImage to eliminate this problem.
  7. Freeing the stream after bitmap creation is perfectly fine. In fact, that's probably what the extra 3 references are for.
  8. If you are unsure whether a stream is being controlled by a GDI+ Bitmap, just clone the stream anyways, and reset the stream pointer (to prevent the inheritance of a random seek position #2). Use IStream::Clone. Then call GdipCreateBitmapFromStream.
  9. GdipCreateBitmapFromStream is a lazy wrapper. Image data is copied when:
    • LockBits wis called with the read flag set
    • When being drawn on via pGraphics
    • in any other case when the file needs to be read.

Q: Does GdipCreateBitmapFromStream advance the seek pointer? (Proof of #2)

A: For some unknown reason, GdipCreateBitmapFromStream will ignore the seek pointer, and then set the stream to position 4096 without fail.

ImagePut.gdiplusStartup()
image := ImagePutStream("https://i.pinimg.com/474x/9a/ba/60/9aba6040f5c0af8c93b388f5df24c121.jpg")
MsgBox (ComCall(5, image, "uint64", 0, "uint", 2, "uint64*", &current:=0), current), "intial"

DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", image, "ptr*", &pBitmap:=0)
ComCall(5, image, "uint64", 0, "uint", 1, "uint64*", &current:=0)
ImagePutWindow(pBitmap, current)

DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", image, "ptr*", &pBitmap:=0)
ComCall(5, image, "uint64", 0, "uint", 1, "uint64*", &current:=0)
ImagePutWindow(pBitmap, current)

Q: Is it bad to create multiple bitmaps from the same stream? (Proof of #5)

A: Yes, it is wrong. Each bitmap after the first contains corrupted image data.

#Requires AutoHotkey v2
ImagePut.gdiplusstartup()
pStream := ImagePutStream("https://picsum.photos/1000.jpg")
ImagePutWindow(pStream) ; Original Image
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", &pBitmap1:=0)
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", &pBitmap2:=0) ; Corrupted
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", &pBitmap3:=0) ; Corrupted
ImagePut.Show(pBitmap1, "1") ; ImagePut.Show() does not clone the bitmap.
ImagePut.Show(pBitmap2, "2")
ImagePut.Show(pBitmap3, "3")
; The corrupted image will be blurry. You will have to zoom in to see pixelation.

Q: Can cloning a bitmap fix the problems shown above? (Proof of #6)

A: Yes. Creating a bitmap clone using GdipCloneImage is considered to be best practice when dealing with a pBitmap of unknown provenance. So long as the backing stream is still alive, a clone of the bitmap will reread the stream and erase corruption.

#Requires AutoHotkey v2
ImagePut.gdiplusStartup()
pStream := ImagePutStream("https://picsum.photos/500")
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", &pBitmap1:=0)
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", &pBitmap2:=0) ; Corrupted
DllCall("gdiplus\GdipCreateBitmapFromStream", "ptr", pStream, "ptr*", &pBitmap3:=0) ; Corrupted
hwnd1 := ImagePut.Show(pBitmap1, "1") ; ImagePut.Show() does not clone the bitmap.
hwnd2 := ImagePut.Show(pBitmap2, "2")
hwnd3 := ImagePut.Show(pBitmap3, "3")
DllCall("gdiplus\GdipCloneImage", "ptr", pBitmap1, "ptr*", &pClone1:=0)
DllCall("gdiplus\GdipCloneImage", "ptr", pBitmap2, "ptr*", &pClone2:=0)
DllCall("gdiplus\GdipCloneImage", "ptr", pBitmap3, "ptr*", &pClone3:=0)
hwnd4 := ImagePut.Show(pClone1, "Clone 1") ; ImagePut.put_window() does not clone the bitmap.
hwnd5 := ImagePut.Show(pClone2, "Clone 2")
hwnd6 := ImagePut.Show(pClone3, "Clone 3")
; The corrupted image will be blurry. You will have to zoom in to see pixelation.
MsgBox "Are Images Equal? " (ImageEqual(hwnd1, hwnd2, hwnd3) ? "True" : "False")
MsgBox "Are Clones Equal? " (ImageEqual(hwnd4, hwnd5, hwnd6) ? "True" : "False")