PNG or JPG (not rgb) over websocket with ArrayBuffer without base64

Is there a way to render a PNG image to canvas without having to encode it to base64?

Server sends a PNG in binary, client receives it in an ArrayBuffer and displays it on the canvas. Only way I could get this to work is by encoding the data to base64 - on the server side - as I need it to be fast. On the client side, I created an image obj with data:image/png;base64 tag.

I know you can create a blob and a file reader but I could not get that to work.

This is the blob version:

var blob  = new Blob([image.buffer],{type: "image/png"});
var imgReader = new FileReader();

imgReader.onload = function (e) {
  var img = new Image();
  img.onload = function (e) {
    console.log("PNG Loaded");
    ctx.drawImage(img, left, top);
    window.URL.revokeObjectURL(img.src);    
    img = null;  
  };

  img.onerror = img.onabort = function () {         
    img = null;
  };
  img.src = e.target.result;              
  imgReader = null;
}
imgReader.readAsDataURL(blob);  

image is Uint8Array. I create a blob from it. The rest is self-explanatory.

Images are correct and valid PNG images. When I send it from the server, I wrote them to a file on the server side and they render fine with an image viewer.