Build Your Own Little Memex with Babashka

..

Slides: Build your own little memex with babashka.pdf. Originally presented on babashka-conf, 2023-07-10.

References—a companion to the slides and the recording.

Intended usage: keep this companion next to the recording or the slides PDF.

Big thanks (incomplete list): Visakan Veerasamy, David Deutsch, Rich Hickey, Ted Nelson, Tim Berners Lee, Ward Cunningham, Niklas Luhman, Sindre, Tore, Steinar Barstad, Michiel Borkent, Jack Rusher, Martin Kavalar, Daniel Slutsky, Alex Miller, Christian Johansen, Erik Assum, Magnar Sveen, Henrik Lissner, Conor White-Sullivan, Jessica Kerr, babashka-conf organizers.

[interlude] List of good things (subjective). (1) Knowing where your ideas come from. Influences, references. ⚓

Build your own little memex with Babashka. Teodor Heggelund. https://teod.eu, @teodorlu.

References + slides are right here; you’re looking at them!.

Today: (PART 1) What is a memex? A brief history of ideas. (PART 2) How to build your own little memex with Babashka. (PART 3) Memexes applied: a tool for exploration

(PART 1) What is a memex? A brief history of ideas.

Information capture, information storage and information retrieval. As We May Think (Vannevar Bush, 1945).

Pieces of paper linking to pieces of paper. Niklas Luhman. Slipboxes, Zettelkasten. 1952-

Virtual pieces of paper linking to other virtual pieces of paper on a computer. Ted Nelson, Hypertext, 1963.

Virtual pieces of paper linking to other virtual pieces of paper on a network describing physics experiments on a network of computers at CERN. Tim Berners Lee, 1990. HTTP and the Internet.

but, do you really want to write <!doctype html><html><head><title>Build yourself a little memex with Babashka ... by hand?

Emerging tools for thought enable frictionless capture and frictionless linking under the label of Personal knowledge management. Org-mode (2003), TiddlyWiki (2004), Roam Research (2019?), Org-roam (2020?).

In addition to frictionless capture and frictionless linking, I want frictionless sharing and frictionless extension with code. Let’s use the web, files and Babashka to move from personal to collective knowledge management!

But the future is already here. Blogs, newsletters, Wikipedia and Clojure Deref are already memexes. Clojure deref is a memex:

memex function clojure-deref mechanism
information capture Alex Miller collects content
information storage https://clojure.org/news/news
information retrieval people read & watch content

Prior work for a babashka memex. I started out with an Org-roam system. Org-Roam is built on Emacs and Emacs Lisp.

Then I built play.teod.eu on that Org-roam foundation. A generated makefile to build the pages. Pandoc for converting documents to HTML and resolving links to slugs. Babashka to stitch it all together. But this system is too complex and has too many dependencies. I wouldn’t recommend this architecture to others who are just starting out. Also, I want an approachable memex toolkit for non-Emacs users.

Desired memex properties (subjective):

  1. Memex is built out of simple pieces
  2. Memex supports multiple source formats: Org-mode, Markdown, Clojure namespaces, …
  3. Memex functionality can be extended with Clojure
  4. Nice editor integrations are provided: VSCode, Emacs, …

[interlude] List of good things (subjective). (1) References. ⚓ (2) Trees. 🌲 (3) Moving water. 🌊

(PART 2) How to build your own little memex with Babashka.

Memex operations:

(make-doc memex uuid slug title)
(make-link memex uuid)
(docs memex)
(->html-files memex)

A memex architecture. Layers:

  1. Editor plugins: VSCode, Emacs, … Editor plugins shell out to a babashka memex CLI.
  2. Babashka memex CLI calls babashka memex lib.
  3. Babashka memex lib uses Pandoc for document conversion. Memex lib works on plaintext files on disk.
  4. Plaintext files are in different formats: org-mode, markdown, asciidoc, clojure namespace with markdown, …

What I’m working on

experimental_memex/
  api.clj       ; ~100 lines, heavily commented
  contrib.clj   ; Quickly collect ideas, append-only.
                ; Empty, as of 2023-06-14
  cli.clj       ; A babashka CLI.
                ; Empty, as of 2023-06-14.

demo. See the old, complex system in action. The new, simple system doesn’t work yet. Contributions welcome, beta testers welcome.

[interlude] List of good things (subjective). (1) References. ⚓ (2) Trees. 🌲 (3) Moving water. 🌊 (4) Exploring your curiosity. 🤔 (5) Cultivating your taste 🖼

(PART 3) Memexes applied: a tool for exploration.

What are you curious about?

What’s good? What do you like?

Give it a title. Put it in your memex. Start exploring!

Want to build a babashka toolkit for building memexes together? Join #memexes on the Clojurians Slack! (renamed from #little-memexes)

This talk focused on theory building. Up next, Michiel Borkent will tackle theory and practice in growing the Babashka ecosystem.

Thank you! Organizers (Paulus, Agatha and Paulus) and sponsors (Pitch, Doctronic, Scarlet, Juxt): you made this happen. Clojure community: You’re awesome. borkdude: for infinite patience with Neil pull requests.

@teodorlu (Github), @TeodorHeggelund (Twitter), #memexes on the Clojurians Slack (renamed from #little-memexes), https://teod.eu/

(APPENDIX) Recommended resources.

Do you want to dig even deeper? Here are some threads you might take a bite of:

  1. The books friendly ambitious nerd and introspect by Visakan Veerasamy. What if we actively explore our curiosity and cultivate our taste in practice? These books give a perspective.
  2. The Beginning of Infinity by David Deutsch. Deutsch gives the best definition of theory that I’ve seen. A good theory explains things well, and a good theory is hard to vary. I believe software engineering and product management should be viewed as disciplines for theory building.
  3. Programming as Theory Building by Peter Naur. That theory can be collected in code, or in a memex.
  4. The Origins of Opera and the Future of Programming by Jessica Kerr. Jessica defines symmathesy as “a learning system made of learning parts”, attributing the term to Nora Bateson. A memex is a great way to collect the written part of the learning system.
  5. Stop Writing Dead Programs by Jack Rusher. I claim that “Memex functionality can be extended with Clojure” is a good thing. Why? Because I believe that the text and the system for working with & viewing the text should be developed together. In my interpretation, Jack Rusher argues for the same thing, “program” is the “text”.
  6. Programming with Hand Tools by Tim Ewald and Running With Scissors: Live Coding With Data by Stuart Halloway. Further argumentation for the importance of being able to change our tools for working as we are working.
  7. Two examples of good explanations: But what is a Fourier series? From heat flow to drawing with circles by 3Blue1Brown and Rule 30 🕹 by Martin Kavalar.

please ignore—early notes

You probably don’t want to keep reading. The above this line, there is polish. Below this line, there is no polish. It contains earliy drafts that didn’t make it into the talk.

Script for demo during presentation

  1. Demonstrate old, complicated system

    1. Create a new page for Niklas Luhman (niklas-luhman) (workspace 2)

    2. Refer to him as

      Sociologist, worked with linked data before computers

    3. Link to Niklas Luhman from this page

  2. Open new, experimental system

    1. https://github.com/teodorlu/lemex.experimental/settings

    2. Change visibility to public

proposal notes 2

Literature review.

Time for doing.

Key ideas.

  1. Moldable message, moldable medium, moldbable tools. Live creation.

Talk outline. Note: I expect that I’ll need to cut scope to stick to 20 minutes, but I think I’ll keep the rough 3 part outline.

  1. Part one: Tools for thinking and the web for sharing.
  2. Part two: My learning practice
    1. Share how I approach learning a new topic
      1. Exploring my curiosity
      2. Making that curiosity explicit, and making it stick.
      3. Sticky application: journaling.
      4. Sticky application: contributing to open source (optionally what I’ve learned from contributing to Neil)
  3. Part three: Build Your Own Little Memex with Babashka
    1. Show how Clojure programmers can currently leverage Babashka and Pandoc for building and sharing their knowledge.
    2. Motivate why
    3. Invite collaborators for writing clojure+babashka libraries for making this happen.
    4. Towards a shared Clojure memex composed of tiny personal islands

Intended talk outcomes.

  1. Audience considers, perhaps wants to use Babashka to learn & share on the internet
  2. Audience sees how to make this happen using tools that already exists
  3. Audience members may consider contributing to an effort / to libraries that make it easier for Clojure programmers to build their own little memex with babashka on the Internet

talk proposal to babashka conf

In May 2022, I decided that I wanted to build my own playful place on the internet. https://play.teod.eu/ was born. I wanted to trust my own choices completely, and make every decision according to my own sense of aesthetics. My previous web site had collapsed under its own weight, and could no longer build, becase of breaking changes in some packages that I used.

I had previously gotten interested in the personal knowledge management movement, primarily Roam Research. I wanted the best parts of Roam with publishing to the web. But I wanted total control. For a time, I explored exporting from Roam to HTML. But I quickly realized that while I loved using Roam’s outlining, it didn’t give me the control I wanted for the web.

A year ago, this effort felt like madness. Today, I feel like it’s one of the better decisions I’ve made in my life. Counted today, I’ve got 12900 lines of org-mode content:

$ cd ~/dev/teodorlu/play.teod.eu
$ find . -iname '*.org' | xargs cat | wc -l
12900

In comparison, my last site got less content:

$ cd ../teodorheggelund.com
$ find . -iname '*.org' | xargs cat | wc -l
1476

I sometimes quote stuff from others, and sometimes copy things around. But I’d guess about 80 % of it is words I’ve written. So something has worked out.

I want to:

  1. Share my motivation for writing on the web
  2. Share what “clicked” for me, and explore why I’ve written about 8 times as much content as I ever wrote on my old site.
  3. Share my efforts to pull out some code that can be reused, and how that’s going.

key ideas

applied curiosity: my motivation for writing.

charity, grace and joy: sharing my work with others, and how that’s going.

Other people are curious too! But the world is filled with too much information, so why should you add any more? And how should you share your work with others? My best bet per 2023-04-22: structure your work as narrow theories. In Clojure, we have a culture for writing narrow libraries. A narrow library solves one thing well with minimal dependencies, and minimal impact on your code. I want to argue that the same goes for knowledge. A good article presents and

TODO

reference collection

copy-pasted things from other places

copied from my personal journal

From Teodor’s public-personal journal, 2023-04-22

  1. the memex is here: it is the world wide web

    1. information deserves to be free
    2. put knowledge in bite-sized definitions (theories) + motivation about why one should care about this
    3. compose bite-sized definitions with playlists – a playlist is an ordered list of links

    Done!

    Q: what’s the right interface for making microtheories?

    Q: what’s the right interface for making knowledge playlists?

    Microtheories need to be good explanations. Examples of good explanations:

  2. it’s worthwhile to build your own little memex

    a “little memex” is your own corner of knowledge. you curate the index. you create and link your nodes. you can refer to and build on other memexes, but the act building the memex is the reward in itself.

  3. your memex should live on the web

    microtheories deserve its own URL. others should be able to leverage your little memex.

  4. your memex is composed of microtheories and knowledge playlists

    a microtheory is a bite-sided deinfition, and motivation for why it matters.

    a knowledge playlist is an ordered list of microtheories.

  5. the act of building your own little memex is to explore your curiosity and journal about your experience.

    1. what are you curious about?
      1. write that question down.
      2. Breathe!
    2. find an answer
      1. write that answer down.
      2. Breathe!
    3. and an action
      1. write that action down.
      2. Breathe!
    4. Listen to yourself.
      1. Breathe.
      2. Do you want to do the action?
      3. if yes, consider doing it.

    Go!

  6. build your own little memex with babashka

    yes!