Covariance Matrix Python - Omit -9999 Value

I'm trying to calculate the co-variance matrix of two completely overlapping images using python. The code for the same is:

stacked = np.vstack((image1.ravel(),image2.ravel()))
np.cov(stacked)

• 
  

  The issue with using this method is that sometimes the images may contain a NoData value like -9999 signifying that the pixel value isn't present. In such a case the np.cov still considers the value causing the mean of the images to drastically vary giving the wrong covariance output.

  
  


• 
  

  If I try to remove the NoData there comes the issue of dimensionality wherein both the images don't have the same dimensions and hence the covariance matrix cannot be computed.

  
  


• 
  

  Manual computation would be highly time consuming

  
  

Is there a value to overcome the issue of NoData and calculate the covariance matrix correctly?