Python - Changing An Image's Contrast With For-Loop

I'm trying to create a for-loop function that will run through every pixel to add contrast to the image. I think I'm close, but right now the image only brightens. If possible, try your best to stick closely to what I have formulated already (FYI, I'm trying to avoid libraries like OpenCV). Thanks for any contributions.

def contrast(img):
   for x in range(img.size[0]):
       for y in range(img.size[1]):
           if (x, y) > 128:
              (r, g, b) = img.getpixel((x, y))  
              img.putpixel((x, y), (r+80, g+80, b+80))  
           else:
              if(x, y) < 128:
                 (r, g, b) = img.getpixel((x, y))  
                 img.putpixel((x, y), (r-80, g-80, b-80))