Parameter is not valid C# CSOM Bitmap Image (Remote Event Receiver) Provider Hosted App

Hi

Has anyone encounter this issue - can't see anything on the net related to SharePoint RER, i'm trying to load an image into stream. This code work fine in SharePoint hosted app, but in provider hosted app, which i'm using for a Remote Event Receiver app, i keep getting "Parameter is not valid" no matter what method i try. My method is below. 

using (FileInformation fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(context, fileRef))
                                    {
                                        using (System.IO.MemoryStream ms = new System.IO.MemoryStream())
                                        {
                                            Helper.CopyStream(fileInfo.Stream, ms);

                                            ms.Seek(0, System.IO.SeekOrigin.Begin);

                                            //using (Image bitmap = (Bitmap)Image.FromStream(ms))
                                            using (Bitmap bitmap = new Bitmap(ms))
                                            {
                                                //do stuff...
                                            }

                                        }
}

I just cannot get pass this line: 

using (Bitmap bitmap = new Bitmap(ms))

This does not work either:

using (Image bitmap = (Bitmap)Image.FromStream(ms))

Could it be this is due to memory allocation, should/could i increase memory if this is the issue?

Thanks