Daniel Tubb

Writer’s Diary #15: Programming Challenge: Ordering a book of fragments?

Daniel Tubb

An update on the beach idea. It works, I think.

What was it? What problem does it solve?

I have a draft of an incomplete manuscript in Tinderbox: a book of ethnographic shorts. The idea? Get a sense of a place and time along the vein of Kathleen Stewart’s Ordinary Affects. It’s got some gems, but the whole draft has been stalled.

The problem? How to order the fragments. 255 fragments, 85,000 words. I do not know how to organize them all. The problem, trying to think about the whole, which is too much.

Some fragments I’ve spent weeks polishing. Others are very rough. But, every time I’ve tried to get into it, the whole seems further away.

I’ve tried various solutions: randomizing notes, linking them and following the links, reading and reorganizing them, etc. But, it’s still too much. The question of order has impeded the actual work that is required. I could probably edit a short to make it good in about 30 minutes. Do that a hundred times over a hundred days, and I have a book. But the second I try to do the whole thing, it falls apart.

A few days ago, while walking on the beach after reading Word Virus, an anthology of William S. Burroughs’ work, I had an idea for a twenty-first century cut-up method using ChatGPT?

At the time, I didn’t quite understand the cut up method, in its nuts and bolts. But, suffice, it’s a way to bring in the random to the writing. More on that tomorrow.

My question on the beach.

Could ChatGPT order the fragments into a book?

I tested the technique with ten fragments, by hand. The results were intriguing enough to make me think it could work.

Round 1: Doing it by hand

What did I do?

First, I asked ChatGPT to create a TITLE, TAGS, and SUMMARY for ten fragments, and return it as a spreadsheet.

Then I asked it to order them.

You are my writing assistant. Help me organize the text fragments. I am using the cut-up method of William S. Burroughs. So I will give you a CSV spreadsheet with a TITLE, TAGS, and SUMMARY. Please give me back a better order. This is for an ethnography. It’s a book of fragments. All I want is the order you suggest. And then a paragraph explaining the order. No need to give a summary or anything. Just the order, then a short paragraph explaining why.

After some tinkering, ChatGPT spit out a plausible order. I tried it out and liked the result.

Round 2: How to do it programmatically

The challenge, how to do this with 258 fragments? The answer, mostly programmatically. But the devil is in the details.

So, first in Tinderbox, I connected Tinderbox to ChatGPT via a Python command line script I’d written in the Spring, with the aid of ChatGPT. After a morning of work, I am able to ask Tinderbox to use ChatGPT to generate a $SubTitle, $Summary, $Tags, and $WritingStyle attribute for each note, automatically.

It took a while for all the agents to run.

But, after playing catch with the kids for a bit, I came back, and each fragment had these updated attributes. (There are some bugs I need to fix. But it’s a personal project, so I’ll just fix them by hand.

What I did next was export a subset of these to a CSV file, using the $ID to keep track of each fragment. I did twenty or so.

I then asked ChatGPT to work its magic.

It returned an order of IDs and an explanation. It seemed plausible. The order was intriguing, and frankly, better than I had. It’s good enough for me to want to keep working on it.

The next challenge was how to import the new order back into Tinderbox.

Round 3: How to reorder notes in Tinderbox

What’s the situation? I have a Tinderbox document with a draft container comprising all the fragments. I have a list generated by ChatGPT of a new order, comprising the $ID.

The question? How to get the latter back into the former so that the fragments were reorganized.

Since everything is a hammer, e.g. use the tools you know, my first thought was to use agents. I could create an agent that took as its own name a search query, and I could use that to find each note, and then update the $Order for each note from the text of the agent. But it felt clunky. Was there a better way?

Dicitonaries. A dictionary can hold a series of paired values:

$MyDictionary=dictionary(“cat:animal; dog:animal; rock: mineral”);

I’ve never used them. But, why not?
It worked. First, I used ChatGPT’s output to create a list in the form of $ID:$Outline.
1:1688302519; 2:1688301878; 3:1688301879; 4:1688301880; 5:1688301899; 6:1688301893; 7:1688302517; 8:1688301881; 9:1688301887; 10:1688301886; 11: 1688301888; 12:1688301890; 13:1688301891; 14:1688301892; 15:1688301894; 16:1688301895; 17:1688301896; 18:1688301897; 19:1688301898; 20:1688301900; 21: 1688301901; 22:1688301902; 23:1688301903; 24:1688301904; 25:1688301905; 26:1688301906; 27:1688301907; 28:1688301908; 29:1688301909; 30:1688301910; 31: 1688301911; 32:1688301912; 33:1688301913; 34:1688301914; 35:1688301883; 36:1688301884; 37:1688301885; 38:1688301882; 39:1688301889; 40:1688301900;

Then I use the following function to reorder the notes:

function fChatGPTReorderNotes(){ var:string vIDString
var:string vIDString;
var:integer vOutline;
var:string vPath;
$MyString=;
$MyDictionary=;

$MyDictionary=dictionary($Text);
$MyDictionary.keys.each(x){ vOutline = x
vOutline = x;
vIDString = $MyDictionary[x];
vPath = $Path(vIDString);
$ChatGPTOutline(vPath)=vOutline;
$MyString=$MyString+vOutline+":"+vIDString + “ [“ + vPath + “]; “;

};
};

That is, the function updates the $ChatGPTOutline for each note, using the imported dictionary held in the $Text field of a note.

I then simply run an agent that searches for ChatGPTOutline0, and is sorted by the ChatGPTOutline. Bingo, I’ve got the book of fragments in the right order.

I now have the fragments, reordered according to ChatGPT’s recommendation. That’s good. The challenge now is how to reorganize all 258 notes.

Round 4: 258 Notes

Here, it’s tricky. ChatGPT has limited memory and can’t iterate on 258 items at once. It seems to be able to do about 20 to 40 at a time. I see three ways to approach this.

  1. Try to get access to ChatGPT’s gpt-4-32k-0613 model which has the ability to work with 32,768 tokens. I’ve signed up for the waiting list. But, I suspect I’m small potatoes.
  2. Use ChatGPT to create a list of 250 notes, but in which each row is much shorter, with ten tokens per line at most.
  3. Programmatically iterate through the all the fragments, comparing each one to the next, and asking what order they should go in. This would be a bit like Tinderbox’s dance feature. The order would evolve over time. I could probably safely do five at once and then run it via agents. I’m not sure, but this is probably the right answer.

In any case, since #1 is beyond my control and #3 is a harder programming challenge, and I’m running out of steam, I’m going to give #2 a try. Cut down the data to something very short. I suspect this means it will be not as a strong a result. But, at this point, the goal is less perfection, and more does this work. I can then come back and try to do the third option another time.

Suffice to say, if you can get the list short enough, it works. Is the list useful, I’m not sure. But, I’m going to try it out, and then get to writing,

Writer’s Diary #14: Wait, Slow Down, You Move Too Fast

Daniel Tubb

Sorry for the riff on a remembered song lyric, but I’ve been thinking all day about waiting, slowing down, and not rushing as part of writing.

Let me take a step back. Yesterday, on the beach, I had an idea—a good one; I think. The cut-up method, a la William S. Burroughs, but modified for the 21st century.

(Or maybe it’s a terrible idea.) I don’t know yet.

In either case, it involves some programming magic in Tinderbox. But, I’m not a programmer, I’m a writer, so it’s going to take some time to make this idea work. A bit of fiddling, trial and error, and time.

What I would do in this situation is start working on this project and drop everything else. I would work so hard, and then about 2/3 of the way through, I would burn out, stop, and never finish. I would get grumpy, and not be nice to be around.

This morning, however, I only had about 30 minutes before I had to pack up for a beautiful day at the beach.

So, I finished the first step, and got far enough to know it would work, and then I realized that the way to move forward would be to work on it a little at a time, rather than all at once. Instead of my quotidian unfinished big blitz, the way forward would be to slow down, and work, and then stop in the middle while the going was good.

Tomorrow I’ll figure out what the steps would be to finish the idea, and then I’ll work on it again the next day. Either I’ll figure out it’s a stupid idea, or it’s an idea worth working on.

Although I’m only a novice runner, I think running is similar. It’s pretty clear you can’t train at full sprint all the time. You’ll burn out. Writing is the same way. Sometimes writing is a marathon, especially at the end. But other times it’s just a slow recovery run.

The trick, sometimes, is to hold off and slow down.

Writer’s Diary #13: Make Writing Fun, Again?

Daniel Tubb

My sabbatical starts tomorrow, so I have some time to get some work done. It feels like I can focus on the Makeshift draft. Yesterday, though, I tried to work on it all day, and after two hours I lost steam. Got grumpy. Tried to push through. Got nowhere.

I’ve been writing this journal for a few weeks now, and I’ve found that part of the trick is to make it a habit. It takes about 30 minutes to write a post, and I’ve been doing it. It’s been fun. I’ve been able to do this and also work on Makeshift. I’ve been able to do this, because it’s a different voice and because it’s plain old fun.

Which leads to me to ask: Can writing be fun?

But I’ve been thinking about how I tried to do too much on the book yesterday. At some point, my focus left me. In previous projects, I would work harder—more coffee, more time, and more effort. But this morning, as I leave for vacation and wait at the mall for family to finish shopping, I wonder if I should borrow a technique from Alan MacFarlane and David Graeber.

Graeber, an anarchist anthropologist who died in September 2020, once tweeted that he worked by having multiple projects on the go. (Of course, I might have misremembered, as I can’t find the tweet. But no matter.) Graber had the thing he had to do (Project A), the thing he wanted to do and was procrastinating on Project A by doing (Project B), and then all the things he didn’t want to do but had to do, e.g. reference letters. I may have misremembered all this, but the gist of it is that he would cycle between projects when one got hard. I like to think, when one got not fun. But, maybe I’m wrong.

Alan MacFarlane has an excellent video on all of this, and talks about doing the easy stuff first—I think he’s drawing on C. S. Lewis’s comparison between writing and eating fish. (The trick, do the easy stuff first.)

What about doing the fun stuff first? What about only doing the fun stuff?

Is it possible to get serious writing done by only doing the project when it’s fun? I’m not sure. But, I’m going to try it for a while.

For now, I’m going to set aside the mornings for writing. But instead of feeling obligated to write only one thing, if one project gets hard, I’ll switch to another. Making writing a chore just leads to grumpiness. And I don’t want to be another grumpy middle-aged white man. There are enough of those.

Writer’s Diary #12: Finishing a Draft

Daniel Tubb

Yesterday I finished a draft of the Introduction, printed it, and re-read it with an eye for editing. Now, with a draft, the question is what’s next? Bits are great. Parts suck. The chapter is unfinished. But, I need a change. The question for the diary: should I stay with it or should I go?

In my experience, writing is iterative work: slow, fast; hard, soft; work, pause. Work hard then leave a piece for a week or a month or more and come back to it.

This morning, my job is to do the big picture carving and then the little editing. Both, the micro and the macro, but the whole thing. With hard, focused work on the whole, and the removal of the big structural pieces I leave the entire chapter ready of a time, and then I can do something else.

Finishing is hard, but I never finish until much later. But along the way, I need space so that when I come back I can see ways to make the words better, shorter, tighter, cleaner, and clearer.

For this, distance is necessary.

Today is the last day of the introduction.

Writer’s Diary #11: The Synapses of Writing

Daniel Tubb

As part of moving this blog to WordPress, I’m going through old posts. This one, from 22 November 2013, stuck with me:

Last night at a party, I chatted writing and rewriting with a friend visiting from Vancouver. She freelances as a copy editor. I told her about On Writing Well, and she echoed many of Zinsser’s suggestions—be short, use as few words as possible, and revise. She suggested an approach new to me: Look for hidden verbs by getting rid of ‘To Be.’ This morning revising a section on small-scale mining, I find the technique works well.

So, what is writing, at least for me, if it isn’t an exercise in applying various tricks and techniques like this? Revising to remove the verb ‘to be,’ removing the first person, eliminating the passive voice, cutting words, revising again, cutting, shortening—these are all tools.

Writing a first draft is one thing. But turning that first draft into something that sings and stringing it together with other texts into a longer piece is another. Part of the trick is the idea and theory. But, much of the trick, at least in the way I approach matters, is an exercise in the not-so-systematic application of a whole series of ticks and tricks.

Some I know and can explain, some are intuitive, some are embodied, and some are technical. My writing style, if I have one, seems to me to emerge in part from the application of these different techniques to writing. For example, a few days ago, I wrote about using online grammar tools—the serial application of tools: cut, edit out, rephrase, rewrite, add things, expand, read aloud, print out and mark up with a pencil. Over time, as the

Parul Sehgal describes George Saunders’ notes on writing in A Swim in a Pond in the Rain:

I’m making the book sound revoltingly technical. It isn’t. Saunders lives in the synapses—he looks at all the minute and meaningful decisions that produce a sentence, a paragraph, a convincing character. He offers one of the most accurate and beautiful depictions of what it is like to be inside the mind of the writer that I’ve ever read — that state of heightened alertness, lightning-quick decisions.

These tools and techniques, which I appreciate and use, are my grist for those tiny, meaningful decisions that I deploy in the synapses of writing.

Writer’s Diary #10: Writing is like Flying, You have to throw yourself at the ground and miss

Daniel Tubb

I used to over-plan each chapter—it had to be perfect, do everything, and be worked out in advance.

For Makeshift I’m more modest, each chapter is one thing done well, in 6,000 words or so. Rather than trying to do too much, each chapter is short and does one thing. A scene, linked to analysis, woven into an argument. Together, these will create the book.

The first chapter begins with a scene of me lying on the linoleum, sweating, hot, unsure of how to proceed and trying to work it all out in advance. There is too much information and no easy solution. My mistake was trying to work it out in advance. The solution? Write the book. Do the work. Think on the page.

The trick to writing is not thinking about writing to much. It’s like the trick to flying, that Douglas Adams describes in his Hitchhiker’s Guide to the Galaxy:

“There is an art to flying, or rather a knack. The knack lies in learning how to throw yourself at the ground and miss. … Clearly, it is this second part, the missing, that presents the difficulties.”
—The Guide.

I might rephrase:

There is an art to writing, or rather a knack. The knack lies in learning how to throw yourself at the page and write. … Clearly, it is this second part, the writing, that present the difficulties.

The Guide goes on:

“You have to have your attention suddenly distracted by something else then you’re halfway there, so that you are no longer thinking about falling, or about the ground, or about how much it’s going to hurt if you fail to miss it.”

I suggest:

You have to have your attention suddenly distracted by something else then you’re halfway there, so that you are no longer thinking about writing, or about the book, or about how much it’s going to suck if you fail to finish.

In other words, to write, you absolutely cannot think to much about the fact you’re writing. Writing is something done in the world, embodied, an act best performed without too much cerebral activity.

That’s the essence of the book’s argument: Writing is done, not contemplated.

I think, with some serious dedication this week, there is no reason I can’t finish the chapter, and move onto the next. But, of course, I can’t think about it.

Writer’s Diary #09: On Freewriting a First Draft

Daniel Tubb

For the first time in a long time, I woke up this morning and started writing at 5:30. It felt good. With a cup of coffee in hand, I sat at the kitchen table as the sun came up and the birds sang. I began working on the first draft of some notes that had come to me the day before and the day before that while I was occupied with other tasks.

I had wanted to write these ideas down the day before yesterday, but I let them linger as I went to bed. My concept for the introduction to the book is that writing is an apprenticeship. Just as my first book was an apprenticeship in gold mining, this book is an auto-apprenticeship in writing.

So I sat down at the keyboard and wrote freely, without forethought, without editing. I followed the method Peter Elbow outlines in Writing Without Teachers. At times, I do this longhand, but this morning I used an electronic typewriter, a rather embarrassing and expensive FreeWriter Smart Typewriter by Astrohaus.

After about five minutes, I had a very rough draft—raw material to work with.

Writers often think and talk about writing as a cerebral activity. But I’ve come to see it as a movement between the cerebral and the use of different tools. It’s as much a cerebral process as it is a manual one. Here, tools matter. Gabriel García Márquez, in his memoir Living to Tell the Tale, describes writing as a kind of carpentry, which requires a lot of technique and craft to hide the joinery. It’s a fitting metaphor. Carpenters use tools, and so do I.

I created a draft through freewriting; applied automatic copy and style editing; revised iteratively and intuitively; edited automatically; revised once more iteratively and intuitively.

Specifically, ChatGPT was a copy editor, DeepL Writer’s beta writing app changes some words, then ProWritingAid fixed grammar and style, and then I had a piece of text to revise using George Saunders, Swim in a Pond in the Rain idea of making iterative and intuitive edits on the page. That is, each pass, I made a myriad of small changes, based on gut feelings. After doing this three or four times, I had this draft, which I left it for a few days, before one more pass. I took screenshots, to illustrate the process. Crucially, automatic editors are helpful, but automatic writer are not. There is a distinction, I’ll write about sometime.

Draft #1: Freewrite

I free wrote the following, on a Freewrite Smart Typewriter by Astrohaus, an ridiculously expensive electronic typewriter, which, like a typewriter, has little distractions.

Note the typos.

Draft 1

Draft #2: Use Chat GPT as a copy editor

I pay for ChatGPT 4, and used the following prompt ti copy edit that first drafts. Here, ChatGPT 4 has replaced my own labour of fixing typos.

Imagine yourself as an AI copy editor, proof reader, and substantive editor. and I will provide you with a text between double quotes, and without making substantive changes, can you spell check, grammar check, punctuate, and split into paragraphs at logical places, and include other revisions in the body of the text in square brackets. Please do this using markdown, in a code block: “…”

Since, I am always concerned about ChatGPT inventing text I didn’t write, I compare it text carefully to the original, in BBEdit using the compare text function. The text is the same, but copy edited.

Draft 2

Draft #3: Use DeepL’s Write

There, I turn to DeepL Write’s AI editor, that will suggest stylistic revisions. They’re good.

Draft 3

Draft #4 – ProWriting Aid

Next, I turn to ProWritingAid, and accept all its style and grammar suggestions.

Draft 4

Draft #5 – Revise with a reader over your shoulder

Drawing on George Saunders book, I undertook a process of reading and as I read, revising as I went intuitively and iteratively.

Draft 5

Draft 6 – Automatic Copy Editing

Then back to ChatGPT, DeepL, ProWriting Aid, then revising through iterative intuition.

Draft 6

Draft 7 – Revise intuitively and iteratively

More intuitive revision.

Draft 7

Draft 8 – Automatic Copy Editing

More copy editing.

Draft 8

Draft 9 – Final Pass

Finally, a few days later, I revised, fixed the images, corrected the URLs, and gave it all a few more passes, without any automatic copy editing.

Done.

Writer’s Diary #08: Notes on Becoming a Writer

Daniel Tubb

My notes on Becoming a Writer by Dorothea Brande (English, 192 pages, Tarcher, 1981). First posted on December 12, 2012, and revised June 21, 2023.

It’s five in the morning, with coffee in hand and sleep still on my mind. What to write? How can I encapsulate the feeling of fieldwork in mere words? As anthropologists, writing is our primary task, but sometimes, words are hard to come by. This is my morning ritual, writing twenty pages. We, as academics, are wordsmiths, writers to the core. Our tools of the trade are notebooks, pens and paper, and laptops.

For most of my adult life, I have been trying to articulate my thoughts into words for others. This endeavour has taken different forms: as a University student, sometimes as a newspaper editor, as a blogger, and increasingly, as an anthropologist. We write about research, we discuss the work of others.

As dawn is an hour away, my pen is already engaged in the task of meaning-making. Anna Tsing’s advice to ‘write early and write regularly’ is a ritual I’m trying to adopt. Write at five o’clock in the morning and force yourself to put pen to paper. It’s a process where words flow freely some mornings, but on others, like today, the journey is tortuous and tedious. My words feel irrelevant and, I fear, ordinary. My prose plods anxiously with little substance offered.

One thing that often goes unsaid is that when we begin fieldwork, we are not just becoming anthropologists but also writers. Writers draft the text, employing various tools and tricks of the trade. There are rhythms to our work that make it bearable. Our prose embodies our life, our sweat, and our tears. Writing is what I learned to do in the field, and if research methods are seldom taught, writing even less so.

Our evenings are often spent making sense of the morning, or our mornings trying to combine words to make sense of experiences, rumours, practices, and suspicions. Writing forms the essence of anthropological sense-making. The hermeneutic circle of life is the task of fieldwork. We are, in fact, novelists, albeit constrained by truth, fact, and reality. Our writing tasks are no less challenging.

Last week, I read Dorothea Brande’s Becoming a Writer. Three pieces of advice resonated:

  1. Write early and write every day. Capitalise on the moments of creativity that dawn provides. Cultivate this time, use it judiciously. First thing, every day. Pay close attention to when these morning sessions yield the most, and try to replicate the conditions of those successful ones.

  2. Cultivate a rigorous mind. Not only should you train yourself to write early, but also designate a specific time to write – a time that invites your creative side. As Brande suggests, “Set a time, say 12:30, and write for half an hour.” Regardless of the circumstances, commit to writing.

  3. Avoid consuming other people’s work, discussing your writing, reading the newspaper, watching movies, and going to the theatre. While Brande would likely disapprove of blogs and Twitter as distractions, her point is that writers thrive on an internal dialogue—our conversation with ourselves. This dialogue can be cultivated with mundane and repetitive tasks but can be destroyed by words, read or heard.

The last piece of advice is the hardest to follow. I often find myself reaching for the radio, the internet, a newspaper, or even a cereal box every few hours. However, in the last few weeks, as I have adhered to this advice, the results on my thesis draft have been remarkable.

Writer’s Diary #07: On Starting

Daniel Tubb

How do you start writing each day? Getting started is often the hardest part—whether it is a book, chapter, article, section, paragraph or project. Procrastination can take as much time as the task itself. This is especially true with a writing project.

There is no magic bullet, but I remember learning a lot from reading and rereading Dorothea Brande’s [Becoming a Writer (1934)](https://www.amazon.ca/Becoming-Writer-Dorothea-Brande/dp/9389157196/ref=sr_1_2? crid=1QMUBDD68ZTEB&keywords=dorothea+brandt+becoming+a+writer&qid=1687262706&sprefix=dorothea+brandt+becoming+a+writer%252Caps%252C137&sr=8- 2&_encoding=UTF8&tag=danieltubb-20&linkCode=ur2&linkId=70052a0d7f212c7c23ef20d57c7a06d1&camp=15121&creative=330641). Although it was written in the 1930s, its advice is often timeless.

One is to set aside time to write, and make a commitment to write at that time. It’s a method of being able to sit down and work on a schedule. At times, especially getting going, I’ve found this advice useful. But now I am more convinced that the trick is to write when you sit down to write, and then to make sitting down a habit. A routine. It takes commitment to get back on the page, but it’s a commitment to cultivate the habit of sitting and, as Brande suggests, to write when you set out to write.

This advice is less about scheduling—which, frankly, does not work for me unless other people are involved—and more about the commitment to do the work when the time comes to do the work.

That’s it. Write when it’s time to write and don’t write when it’s not.

Writer’s Diary #06: Writing and Routine

Daniel Tubb

Frankly, a challenge many of us face as writers is finding the time to do the work. Over the past ten days, with travel, students and the urgent tasks that have arisen, it has seemed impossible to find the time.

This morning, however, I am back at this Writer’s Diary. Part of the problem is that there are days when the routine changes drastically. Writing a book, I think, is an exercise in doing the work over and over again.

My father is building a house. He works on it almost every day, most of the day, except at weekends when he does other things. That’s his work. He doesn’t check emails during the day; he doesn’t use the phone; he doesn’t attend meetings; nor does he get pulled in a dozen different directions by emergencies that arise. Instead, he works on the house. The works has its own physicality, rhythm and routine.

As a professor with small children and a plethora of activities, the challenge is to find the time. My father’s method is inspiring. Do the thing. That’s it.