How to create this image effect programatically?
New here? Learn about Bountify and follow @bountify to get notified of new bounties! x

The Task

Pixel sorting is a visual effect created by applying taking the rows, columns, or other set of pixels in an image and ordering them according to some value of those pixels, like how much red is in each pixel, their luminosity, hue, etc. It tends to create really cool visual effects, and it's mostly just for fun and artistic purposes.

I would like to see some function code that efficiently implements an artistic glitching effect called "pixel sorting" found in generative art and computational photography.

Here are some examples of it:

Type 1: Photo Filter

Type 2: Generative

Rules

  • Please use only these tools: Python, Javascript, Mathematica, C/C++
  • Please write a single parameterized function that applies the effect on images (Type 1)
  • For a nice tip, write a function that can convincingly generate random examples these textures (Type 2).

References

Crowdsource coding tasks.

1 Solution


The python solution is surprisingly simple! I added multiple modes to sort by (luminosity, red, green, magenta, cyan, etc.). Should be pretty easy to use, thanks for the challenge!

from PIL import Image
import numpy as np

mode = {
    'sum-rgb': lambda x: x[0] + x[1] + x[2],    # sort by sum of rgb values (grayscale)
    'red': lambda x: x[0],  # sort by red value
    'green': lambda x: x[1],    # sort by green value
    'blue': lambda x: x[2], # sort by blue value
    'yellow': lambda x: x[0] + x[1], # sort by yellow value
    'cyan': lambda x: x[1] + x[2], # sort by cyan value
    'magenta': lambda x: x[0] + x[2], #sort by magenta value
    'luma': lambda x: 0.02126 * x[0] + 0.7152 * x[1] + 0.0722 * x[2] # sort by humar color perception (luminosity)
}

def pixelsort(image_name, mode, row=True, reverse=False):

    # PARAMETERS
    # image_name: name of image file
    # mode: mode to sort by
    # row: sort rows if True, otherwise sort by columns
    # reverse: sort in reverse if True

    picture = Image.open(image_name)

    if row: # convert numpy array to regular python list
        pixels = np.array(picture).tolist()
    else:
        # if you want to sort columns instead of row, just flip the image over its diagonal
        pixels = np.array(picture).transpose().tolist() 

    for y in pixels:
        # sort each row (or column)
        y.sort(key=mode, reverse=reverse)

    pixels = np.asarray(pixels, dtype='uint8')
    if not row:
        # flip back over the diagonal if sorting by columns
        pixels = pixels.transpose() 

    # convert back to image
    im = Image.fromarray(pixels, 'RGB')
    im.show()

    return im

pixelsort('image.jpg', mode['red'], True) # 'sort image's rows by red'

I added comments for further code clarity, hopes this helps :)

Edit: I realize you probably want to save the pixelsorted images, so here's a little function to do that. Let me know if there's anything else I can help with!

def save_as(image, name='sorted.jpg'):
    image.save(name)

image = pixelsort('image.jpg', mode['red'], True) #pixelsort
save_as(image, 'pixelsort.jpg') #save image

Edit 2: Added start and stop heuristics, Edit 3: Added modes to sort by hue, saturation, and lightness

from PIL import Image
import numpy as np


def hue(x):
    r = x[0] / 255
    g = x[1] / 255
    b = x[2] / 255
    Cmax = max(r, g, b)
    Cmin = min(r, g, b)
    delta = Cmax - Cmin

    if delta == 0:
        hue = 0
    elif Cmax == r:
        hue = 60 * (((g-b)/delta)%6)
    elif Cmax == g:
        hue = 60 * ((b-r)/delta + 2)
    else:
        hue = 60 * ((r-g)/delta + 4)
    return hue

def lightness(x):
    r = x[0] / 255
    g = x[1] / 255
    b = x[2] / 255
    Cmax = max(r, g, b)
    Cmin = min(r, g, b)

    return (Cmax + Cmin) / 2

def saturation(x):

    r = x[0] / 255
    g = x[1] / 255
    b = x[2] / 255
    Cmax = max(r, g, b)
    Cmin = min(r, g, b)
    delta = Cmax - Cmin

    if delta == 0:
        sat = 0
    else:
        sat = delta / (1 - abs(2 * lightness(x) - 1))

    return sat


mode = {
    'sum-rgb': lambda x: x[0] + x[1] + x[2],    # sort by sum of rgb values (grayscale)
    'red': lambda x: x[0],  # sort by red value
    'green': lambda x: x[1],    # sort by green value
    'blue': lambda x: x[2], # sort by blue value
    'yellow': lambda x: x[0] + x[1], # sort by yellow value
    'cyan': lambda x: x[1] + x[2], # sort by cyan value
    'magenta': lambda x: x[0] + x[2], #sort by magenta value
    'luma': lambda x: 0.02126 * x[0] + 0.7152 * x[1] + 0.0722 * x[2], # sort by human color perception (luminosity)
    'hue': hue,
    'saturation': saturation,
    'lightness': lightness

}

def pixelsort(image_name, mode, row=True, reverse=False, start=lambda x: False, stop=lambda x: False):

    # PARAMETERS
    # image_name: name of image file
    # mode: mode to sort by
    # row: sort rows if True, otherwise sort by columns
    # reverse: sort in reverse if True

    picture = Image.open(image_name)

    if row: # convert numpy array to regular python list
        pixels = np.array(picture).tolist()
    else:
        # if you want to sort columns instead of row, just flip the image over its diagonal
        pixels = np.array(picture).transpose((1, 0, 2)) 
        print(pixels.shape)
        pixels = pixels.tolist()
    new_pixels = []

    for y in pixels:

        # sort each row (or column)
        index_start = index_of_first(y, 0, start)
        if index_start < 0:
            index_start = 0
        index_stop = index_of_first(y, index_start + 1, stop)
        if index_stop < 0:
            index_stop = len(y)

        segment_to_sort = y[index_start:index_stop]
        segment_to_sort.sort(key=mode, reverse=reverse)
        new_pixels.append(y[:index_start] + segment_to_sort + y[index_stop:])

    new_pixels = np.asarray(new_pixels, dtype='uint8')
    if not row:
        # flip back over the diagonal if sorting by columns
        new_pixels = new_pixels.transpose((1, 0, 2)) 

    # convert back to image
    im = Image.fromarray(new_pixels, 'RGB')
    im.show()

    return im

def index_of_first(arr, index, predicate):
    for i in range(index, len(arr)):
        if predicate(arr[i]):
            return i
    return -1

def save_as(image, name='sorted.jpg'):
    image.save(name)






start = lambda x: x[2] < 100 # start when blue is less than 100
stop = lambda x: x[2] > 100 # stop when blue is greater than 100

image = pixelsort('image.png', mode['sum-rgb'], row=True, reverse=False, start=start, stop=stop)
The result looks really bad :( https://imgur.com/a/t0ewdyI
mr142 3 months ago
Needs better heuristics about when to start and stop and what colors to sort...
mr142 3 months ago
I think you just have to find the right images for this kind of stuff. Here's a random image I just applied to code to and it seems to work pretty well https://github.com/BrianSantoso/images/blob/master/pixelsort/pixelsorted2.jpg I'll add some heuristics as to when to start/stop the sort.
BrianSantoso 3 months ago
But yeah, it really depends on what image you're using and which mode you sort it by. For example, if you had a picture of the ocean but decided to sort it by its red values, then the results would look pretty bad. You can't throw any old image at it and get amazing results, its a form of art as well :D
BrianSantoso 3 months ago
Finished. Added infinitely flexible heuristic as to when to start/stop the sort.
BrianSantoso 3 months ago
I was going to try my hand at a solution, but this solution seems to be what you're looking for. If you want anything better than this, I think you'll have to up your asking price my guy.
mattau13 3 months ago
Hey, this took quite a long time... do you think I could get a tip?
BrianSantoso 3 months ago