dechroma
A while back somebody posted some “amazing” images which were black and white except for the stripes that were colored. So, not black and white, but the point was to demonstrate that vision is highly perceptual and the brain will interpolate from what’s there. I thought this might be fun to play around with. I guess it kinda works, but I think some of the demo images were selected carefully.
I was going to post some of my results, but then I thought it would be more fun to post the generator instead. Evaluate your own results.
dechroma
To recap, the idea is that we can eliminate most of the color information from a photo, leaving only dots or stripes, but it will appear mostly normal. Like an extreme form of chroma subsampling common to compressed jpeg and video files.
There’s a few patterns one can choose. I went with a diagonal stripe, which seemed to give slightly better results when there were horizontal or vertical lines in the image. Then for each region, in my case an 8x8 block of the image, I sum all the saturation information, and redistribute it to only the pixels along the diagonal. In theory, if you see a gray square with a bright yellow line running down the middle, it will appear like a yellow square. In practice, there’s usually too much saturation in the original image, and the output looks kinda washed out.
So, the effect works, but I think the examples that get passed around were selected because the input wasn’t too vibrant. I of course went for the most vibrant images I could find on unsplash. You can tell the effect works, but I don’t think the brain compensates as much as I was led to believe. Or maybe I’m not manipulating the image correctly.
wasm
Once I had the demo working with some files in the filesystem, I thought why not make it cloud powered and stick it in the browser. Fortunately, go can be compiled to wasm objects, and with a few caveats, seems to work. Getting data across the javascript to wasm interface requires a fair bit of boilerplate, but it works, and with a few helper functions, some very basic DOM manipulation is possible from go.
While researching options, I came across shimmer which is a rather similar program that does more traditional image adjustments.
I wanted the demo to be able to load URLs, but you might hurt yourself, so CORS doesn’t allow that. So the dream of running in the browser just like running on your computer remains unrealized. You can run an open CORS relay, but that’s kinda hackish?
It runs. It’s kinda slow I think. Also, the feedback should change to “working” when you start processing an image, but instead nothing happens until it’s done. Not sure how to debug that. The mysteries of the web.
samples
These samples have been resized and recompressed with jpeg, so the chroma has been resampled many times over. They are not at all pixel accurate, but they show approximately the same result as the full size images. You can always input the original images to see what happens. Just don’t pixel peep the below. They’re presented here for the lazy.
Here’s a colorful frog. The colors are recognizable, but we’ve clearly lost some vibrancy.
Here’s a girl holding a delicious whiteclaw. Okay, I guess.
Here’s a storefront window. This one looks pretty good I think. Without the side by side comparison, you wouldn’t immediately recognize that it’s been desaturated.
Another thing to consider is these images were almost certainly retouched in lightroom to push the saturation. More natural images with less pop would probably come out better.
demo
It’s not pretty, but it may work. It’s a few megs of wasm, so you have to load it first.
code
Here it is, such as it is.
I don’t know if this is a great algorithm, or even a good one, but the general idea is to slide a window over blocks of the image collecting, then revisit them. Like a kernel without having to math. It made it easy to tweak and adjust.
package main
import (
"bytes"
"image"
_ "image/jpeg"
"image/png"
"io"
"log"
"net/http"
"os"
"time"
"syscall/js"
"github.com/lucasb-eyer/go-colorful"
)
func loadImage(what string, reader io.Reader) (image.Image, error) {
img, _, err := image.Decode(reader)
if err != nil {
return nil, err
}
return img, nil
}
func loadImageUrl(url string) (image.Image, error) {
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
req.Header.Add("js.fetch:mode", "no-cors")
req.Header.Add("js.fetch:credentials", "omit")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
log.Printf("resp: %s", resp.Status)
return loadImage(url, resp.Body)
}
func loadImageBytes(data []byte) (image.Image, error) {
r := bytes.NewReader(data)
return loadImage("data", r)
}
func loadImageFile(filename string) (image.Image, error) {
fd, err := os.Open(filename)
if err != nil {
return nil, err
}
defer fd.Close()
return loadImage(filename, fd)
}
func saveImage(img image.Image, filename string) error {
fd, err := os.Create(filename)
if err != nil {
return err
}
defer fd.Close()
err = png.Encode(fd, img)
if err != nil {
return err
}
return nil
}
func saveImageBytes(img image.Image) ([]byte, error) {
var buf bytes.Buffer
err := png.Encode(&buf, img)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}
type Window interface {
Stride() (int, int)
Slide(func(x, y int))
Apply(func(x, y int))
Reset()
}
type AdjustWindow struct {
stride int
residue float64
slide func(w *AdjustWindow, x, y int)
apply func(w *AdjustWindow, x, y int)
reset func(w *AdjustWindow)
}
func (w *AdjustWindow) Stride() (int, int) {
return w.stride, w.stride
}
func (w *AdjustWindow) Slide(x, y int) {
w.slide(w, x, y)
}
func (w *AdjustWindow) Apply(x, y int) {
w.apply(w, x, y)
}
func (w *AdjustWindow) Reset() {
w.reset(w)
}
func resaturate(img image.Image) image.Image {
bnds := img.Bounds()
rgb := image.NewRGBA(bnds)
window := AdjustWindow{
stride: 8,
residue: 0.0,
slide: func(w *AdjustWindow, x, y int) {
col, _ := colorful.MakeColor(img.At(x, y))
_, c, _ := col.Hcl()
w.residue += c
},
apply: func(w *AdjustWindow, x, y int) {
col, _ := colorful.MakeColor(img.At(x, y))
h, _, l := col.Hcl()
xx := x % w.stride
yy := y % w.stride
if xx == yy {
amt := w.residue / float64(w.stride)
if amt > 1.0 {
amt = 1.0
}
col = colorful.Hcl(h, amt, l).Clamped()
} else {
col = colorful.Hcl(h, 0.0, l).Clamped()
}
rgb.Set(x, y, col)
},
reset: func(w *AdjustWindow) {
w.residue = 0
},
}
xs, ys := window.Stride()
for y := bnds.Min.Y; y < bnds.Max.Y; y += ys {
for x := bnds.Min.X; x < bnds.Max.X; x += xs {
window.Reset()
for yy := 0; yy < ys; yy++ {
for xx := 0; xx < xs; xx++ {
window.Slide(x+xx, y+yy)
}
}
for yy := 0; yy < ys; yy++ {
for xx := 0; xx < xs; xx++ {
window.Apply(x+xx, y+yy)
}
}
}
}
return rgb
}
func main() {
for {
time.Sleep(100 * time.Second)
log.Printf("a tick")
}
}
func getElement(name string) js.Value {
return js.Global().Get("document").Call("getElementById", name)
}
func jsiCall(name string, args ...interface{}) js.Value {
return js.Global().Call(name, args...)
}
func jsiBytes(data []byte) js.Value {
j := js.Global().Get("Uint8Array").New(len(data))
js.CopyBytesToJS(j, data)
return j
}
func jsiGetBytes(val js.Value) []byte {
buf := make([]byte, val.Get("byteLength").Int())
js.CopyBytesToGo(buf, val)
return buf
}
func jsiDechroma(this js.Value, args []js.Value) interface{} {
getElement("feedback").Set("innerHTML", "working!")
log.Printf("called from js")
bytes := jsiGetBytes(args[0])
go func() {
img, err := loadImageBytes(bytes)
if err != nil {
log.Printf("trouble loading: %s", err)
return
}
img = resaturate(img)
data, err := saveImageBytes(img)
if err != nil {
log.Printf("trouble saving: %s", err)
return
}
jsiCall("updateImage", jsiBytes(data))
getElement("feedback").Set("innerHTML", "done!")
}()
return nil
}
func init() {
cb := js.FuncOf(jsiDechroma)
js.Global().Set("dechroma", cb)
}