Bad Apples

Dearest Rachel –

In the game of Cards Against Humanity that we used to enjoy playing together in large groups, there is a black card that reads “________: Awesome in theory, kind of a mess in practice,” to which the players submit their best white cards with one outlandish concept or another. The point, of course, is to see which suggestion causes the judge for the round to laugh the loudest, blush the deepest, or do whatever will cause them to award the round to them for filling the blank in the best. What I’m dealing with at the moment isn’t anything that would necessarily win the round for me, unless one considers ‘most painfully accurate’ a prize-winning category.

You see, it so happens that I’ve made a few decisions regarding the AI art generator that don’t work nearly as well as expected. I had thought that, by coming up with a checkpoint file of as many pictures of you as I could find, it would allow the computer the maximum variety of poses and expressions to work with. Like the card says, it makes perfect sense in theory – and, if you give it the chance to go through a few iterations, tweaking it as you go, you keep getting closer all the time to what you want (although you’ll notice that, as an example, I couldn’t get you to actually wear the pink sailor fuku I found in that one blue bag in the basement. And that was after several dozen attempts to get more and more specific). For all the fairly instant gratification of a quickly-generated image, the fact that it’s not as easy as you’d think to get exactly what you want from the computer, which can be mildly frustrating.

But that’s not the reason I’m reconsidering what I’ve done with your pictures – at least, not exactly. The thing is, I’d included a few shots that, upon further reflection, were less than optimal for my purposes. In some cases, I knew this, but as I didn’t think I had a lot to work with, I found myself shrugging; what are ya gonna do? Besides, it’s only a photo or two here or there; among over a hundred and eighty, how much difference can they make?

But it turns out that, when the AI generates a picture inspired by one of these bad apples, it’s pretty obvious.

Let me give you an example or two of the most egregious offenders. This first one is something of a less-than-flattering, but interesting, close up pose of you presumably lying on a couch or something.

You’ll notice that the picture cuts off the upper right hand corner of your face. Again, it’s an odd picture, but that’s part of what makes it interesting. At any rate, the software boasts an ability to do what it calls ‘outpainting,’ which is basically tech-speak for guessing what the rest of the picture might look like just outside of the camera’s eye, and continue to draw from there. The human eye does something of the same thing when confronted with an image that’s partially blocked by something in front of it – it’s something that in artistic circles is referred to as ‘closure.’ I, and anyone who knew you, can imagine what the rest of your face looks like, as well as your hair, which hardly appears in this shot.

The computer, however, isn’t nearly so talented…

At least, not as a rule. There are versions where it manages to imagine and draw your full face:

…but as often as not, the computer basically assumes (quite logically, I suppose; which stands to reason for a computer to do) that, since the original photo was cut off oddly like it was, that I want similar pictures to be cut off likewise. Which isn’t true at all, but there’s no easy way to tell the machine any of that. I don’t bother to keep a lot of these individual examples any more – since I usually have the system run a given prompt in batches of nine, it generates a grid of possibilities, leaving me to throw out the ones I don’t like (or just don’t look like you – there’s a lot of very pretty output that I’ve jettisoned just for that reason alone) and keep the grids for later reference should I wish. Needless to say, this pose isn’t kept very often.

The second case is a little less obvious in terms of its origin and deciding whether to keep or discard pictures of you. I have a photo or two of you from a number of years back – I can’t seem to figure out from when, but I know it’s from Christmas…

And it’s that paper crown that the computer seems to absolutely fixate on. Sure, you wore a lot of things in your hair, almost as a matter of course. When it was long (the way I thought it looked best, but which you only let get that way during the winter, as it was too warm in the summer months to wear such a warm cover on your head and neck. The problem is, between convention season and holiday vacations, most of our pictures were taken in summer, when you wore those close-cropped pixie cuts), you pinned it back, and out of your face. The few photos I have of you with your hair let completely loose…

…it just looks weird and unnatural; even I knew to not include this shot in the collection for the computer to work with.

But the computer absolutely goes nuts over this paper crown. It can’t seem to tell if it’s part of your hair, a bathing cap, or whatever, but it knows you liked purple, and includes it (in various forms) in an astonishing number of compositions:

All these, from a single element in a single photo. Heck, I have another shot of you from that same Christmas (at least, you were wearing the same sweater, so I rather assume it was the same one), and you’re not wearing that crown, so it isn’t as if you bothered with it all day, let alone dozens of times in so many situations. But the system literally draws its own conclusions.

With that in mind, I should also mention that, since so many images of you were peripheral to the main subject of the photos (you know, general scenery and architecture of places we never thought we’d get to, and needed to record in order to go over at some undefined point in the future – never thinking that the real images we would want to see again were those of each other), there are a number of pictures of you from behind, or at least, looking away from the camera. I hardly need to tell you what the computer makes of that.

To be honest, there’s a certain metaphorical nature to these results that isn’t entirely out of place. As the song goes, “you smile, and turn away,” so the fact that the computer generates so many such pictures seems almost appropriate; the long- distance shots, even more so.

But that’s not what I really wanted to create in terms of pictures of you. So I may have to scrap (or at least, set aside) this particular aggregation for something else. For all I know, this may be why they recommend only about twenty or thirty pictures when creating these things; and all this time, I assumed it was because otherwise, it would take too much time and processing power to generate.

And in any event, having discovered your parents’ cache of old photographs, combined with some rudimentary understanding of how to extract still photos from our vacation videos, I have a lot more to work with than I thought I did at one point. To be sure, all of these would need some kind of enhancement before including them in a selection of photos to use as a basis, so it’s gonna take some time to assemble. But maybe it would allow me to weed out the bad apples that would interfere with generating a decent picture.

With that in mind, honey, keep an eye on me, on these projects, and wish me luck. I’m going to need it.