play.teod.eu

..

Canonical URL https://play.teod.eu/
Source https://github.com/teodorlu/play.teod.eu
How was it made? talk 1 Build Your Own Little Memex with Babashka
How was it made? talk 2 Clojure visual-tools meeting 15 - play.teod.eu, Kindly

Yeah, the page on play.teod.eu on play.teod.eu. Kind of weird, right?

Whether this is “a remote reference” or not is up for discussion.

:)

I’m using remote references as direct commentary on for example The Beginning of Infinity, and I really like that. Rich references. Mmmmm.

What follows is discussion / a retrospective for this page.

  1. first, a critique synthesis
  2. then, a critique sandbox & options
  3. then, a journal

Language: mostly English, some Norwegian.

“As someone who is not Teodor, should I read this?” This is mostly a project page for myself. So it’s definitely not required reading. But it’s public, and open for comments!

retrospective—synthesis

Attention is aimed here.

play.teod.eu success metric: strong language

I want to use play.teod.eu to power up my language.

Source code listings are BAD

Just look at Recursive document transformations with Pandoc and Clojure.

  1. Hard to disambiguate code listings from text
  2. Code listing syntax highlighting has horrible contrast with the rest of the text.

I feel like I want to ask Oddmund for help / advice.

CSS pain points

I want to learn CSS. I want to use play.teod.eu to learn CSS. As of 2022-08-20, Pandoc controls the CSS, and I don’t have too much leverage.

Examples from other people / inspiration / ideas to steal

See also

I cannot easily customize the CSS without building everything from scratch.

Pandoc provides CSS to me. But I want to configure stuff myself too. Does that mean I have to write all the CSS from scratch?

What is a good first step? Should I understand how the existing CSS works first? Or should I just remove it all and start from the start?

  1. Why just remove it all

    I can be okay with it looking a bit crap. I need to take responsibility for writing the CSS myself I want to become good at CSS at any point.

  2. Why tweak

    Pandoc’s look is good right now. And I can move incrementally. I don’t have to redo everything.

Tables are too narrow

I cannot use tables effectively. They wrap way before they are useful.

Example of a bad table: https://play.teod.eu/git-commit-messages/

the whole site is narrow

Here are two example lines from https://play.teod.eu/git-commit-messages/:

I sometimes write bad commit messages. Here on play.teod.eu, I write particularly bad commit messages. See for yourself:

That’s 68 characters.

Here are some example lines from https://drewdevault.com/2022/07/09/Fediverse-toxicity.html:

Mastodon, inspired by GNU social, together with Pleroma, form the most popular components of what we know as the “Fediverse” today. All of them are, in essence, federated, free software Twitter clones, interoperable with each other via the ActivityPub protocol.

That’s up to 95 characters.

Source code blocks are too narrow

And for source code, Drew DeVault is using an even smaller font. The article Porting Doom to Helios has some source code. Here are some of the longer lines:

static size_t writecons(FILE *f, const unsigned char *buf, size_t size) {
    sys_writecons(f->wbase, f->wpos - f->wbase);
    sys_writecons(buf, size);
    f->wend = f->buf + f->buf_size;
    f->wpos = f->wbase = f->buf;
    return size;
}

I think the source code too is meant to reach about a 100 characters in width.

A shallow link just points somewhere. A reified link carries meaning and metadata.

Cross site linking with org-roam.

So … getting Org-roam to find links here was easy. Just set the root dir, and add ID properties for each page that should be indexed.

Status: links are working in Emacs. Links are not working on the web. Why? I’m using Pandoc to parse org-mode links, and pandoc doesn’t know how to handle id links. Perhaps I can solve this myself with a filter? Hmm.

Options:

Filter architecture draft:

What would the “check in everything” approach mean?

  1. Simply link to folders
  2. Then simply manage that.

I wouldn’t be able to get the nice “straight to file” Org-Roam style navigation. But do I really need that?

I want reified links for play.teod.eu.

  1. Why reified links?

    Because I can model and query knowledge.

    A good first use case is being able to resolve the id: links I can insert with Org-Roam, like this: Feedback loops, interface design and how stuff works.

    1. Challenges

      Can I get the links out with Pandoc JSON and a Clojure filter with a walk? I think so.

      And how should I represent the links?

      Well, there’s not only the links. I need to resolve the links. Hmm, that could mean extracting links. And links could have some kind of ID. What if I simply keep the original link target, and use that as an ID? But … linking to the ID of a page and linking to the path of a page is different.

      ;; here's a link to this page:
      {:id   "658447a3-00e6-44aa-963e-d2f5938c50d3"
       :host "play.teod.eu"
       :path "play.teod.eu-rolling-retrospective"}

      No, actually, that’s not a link. That’s a link target. If I want that as an entity, I’ll need … an ID too? Hmm.

      What can differentiate the links?

      ;; here is this page:
      {:id   "658447a3-00e6-44aa-963e-d2f5938c50d3"
       :host "play.teod.eu"
       :path "play.teod.eu-rolling-retrospective"}
      
      ;; What does the link to this page look like?
      {:target/id "658447a3-00e6-44aa-963e-d2f5938c50d3"
       :source/id "abc123"
       ;; what's link ID?
       ;; what's link type?
       ;; What kind of link types can I have?
       }

      How do we know the difference between one link and another?

      Option:

      • source, target and increasing number
      • just source and target

      Challenge:

      • I don’t control Org-mode’s UI. So I can’t add a link ID there. I probably don’t want to keep track of link IDs that detailed either. As long as I’m using plain text. Or … add a ?referrer=abc123?
    2. referrers not allowed

      Here’s some verbatim org-code:

      I tried linking to id:abc123?referrer=123 rather than id:abc123, but to no avail. How about linking to id:abc123#referrer-123? Nope.

I want to tag each article with :generality :novelty :precision :subjectivity

axis as 0 as 1
:generality event physical law
:novelty known by all known by 1
:precision dream logic
:subjectivity event taste

I believe reified links can model knowledge well. And on top of that model, we can build better UIs.

Here’s what an UI could look like: https://embed.kumu.io/1feca726268dbbda0f905fb7be844e5e#anxiety-driven-procrastination

how can i improve front page attention design?

i want a new category - not indexed

Relevant for example for external references.

so perhaps a :type :reference? :reference-target #{:book :web-essay ,,,}

Cultivate and learn from how others are doing their stuff

https://til.simonwillison.net/sqlite/enabling-wal-mode

Bra / spennende:

  1. Godt eksempel på hvordan man kan vise “ting som linker hit” nederst
  2. Veldig oversiktelig struktur
  3. Simon Wilson strukturerer notatene sine i underkategorier.
    1. Han bruker til.simonwilson.net der jeg bruker play.teod.eu
    2. … men der jeg holder meg til ett suffiks (feks /maksimal-opsjonalitet), bruker han kategori først (sqlite). Så … litt som jeg samler tematiske ting i journaler. Tror ikke han har kategorier i kategorier, men det er i alle fall kategori -> ting.

https://memex.marginalia.nu/

Bra / spennende:

  1. Han bruker siden til aggregering av podcaster. Jeg er interessert i noe sånt for meg selv.
  2. Siden ser ut til å ha login og et redigerings-lag rett i nettleser
  3. Støtter gemini.

Ideas from others

Patrick Dubroy on how to build ideas in a shared space:

Like it's just so much harder to achieve what SPJ describes here with systems work pic.twitter.com/6ShxWQsAOr

— Patrick Dubroy (@dubroy) August 26, 2022

Renaming an article is a pain

Right now:

  1. Rename in index.org
  2. Rename in play.edn
  3. Then run teod-play-refresh (SPC å r)
  4. Then update articles (accidentally) when a link to then is encountered.

Idea:

  1. Reduce the renaming to a single action
  2. Introduce explicit dependency from a page to the other pages which it refers to
  3. When / if I redo the build system (remove Make), I’ll use that opportunity to:
    1. Extract page dependencies to data
    2. Use that data to trigger rebuilds & stuff.

Table of Content is bad

I want to be able to opt out of TOC generation

Per 2022-12-31, I have several pages where I’ve omitted to create headings in order to avoid having a TOC. An example is the Aphorisms page. It has a list of my own aphorisms, and a list of links to other people’s aphorisms. I wanted to link to it from How to solve problems, but I couldn’t link directly to the right section because it wasn’t a header.

Note: I guess I could create an anchor? That’s not a bad idea.

Problems

Solve with Pandoc filter?

Now, I use the –toc pandoc option:

pandoc -s --shift-heading-level-by=1 --toc --from=org+smart -H live.html -i play.teod.eu-retrospective/index.org -o play.teod.eu-retrospective/index.html

If I create my own filter instead, I can do something else.

Something like:

pandoc --from org+smart -i play.teod.eu-retrospective/index.org --to json \
    | ./play.clj transform --generate-toc --link-up  \
    | pandoc --from json -o play.teod.eu-retrospective/index.html

Questions.

How to?

Examples of nicely formatted table of contents:

https://guzey.com/personal/what-should-you-do-with-your-life/#cold-emails-and-twitter

TOC contains “discussion”, “comments” and such

See: 2022-12-19/I want a configurable table of contents.

retrospective—rest

Ledger.

Working with pandoc --standalone is causing problems

As long as I’m using the --standalone stuff, I’m making problems for myself. Example: I loose roundtripping. If I try to roundtrip with --standalone, I get double table of contents and double title. I don’t want that.

Found a walkaround for roundtripping: avoid writing title info

pandoc  \
    --standalone \
    -V title:"" \
    -i index.html \
    --filter rickroll.sh \
    -o rickroll-ourselves.html

Specifically, -V title:"" makes sure the title isn’t set twice. Regardless, I think I want to control the standalone stuff myself.

See also: Table of Content is confusing

I want - site previews

When I post a Twitter link, Slack is able to create a nice preview. When I post a play.teod.eu link, there is no preview.

Solved problems

Bad.

Problem definition

  1. They don’t link to the right place
  2. They even link to id:… things that don’t work

Hmm, prehaps href is better. 🤔

2022-08-19 - solved.

All org documents now get their links rewritten. The link rewrite filter is written so that it only rewrites the links it should rewrite.

I need to “contract” / “narrow” before publishing

Problem statement — how can I use this site to grow messy stuff AND share reasonable stuff with others?

Early view

The stuff that’s published on my site right now is a real mess. Mess how? In many ways. But the big one: the ideas are intertwined. There’s coupling. I could remove that coupling. Or at least attempt to remove that coupling.

2022-07-12 says hello

At the time that I wrote the text above, everything was one big lump.

Merely splitting between “feedback welcome”, “forever incomplete” and “mess” helps me A LOT personally.

2022-07-14 says hello

I’ve recently published two small things on the Clojurians Slack. First a bit about Emacs — how to use read-string, completing-read and let* to write tiny UIs. Then a Interaction value differential page.

Am I happy with the results?

Yes.

Very much, actually. Every publish-action gave me some nice interactions. In the Emacs case, I simply copied from my notes and Emacs config and wrote a small, little Slack message. In the Interaction value differential article, I published something. And I liked what i published.

😊

2022-08-20 consid er this solved!

I’ve cleanly separated between stuff that people should consider reading, and stuff that people should ignore.

Creating new pages

Currently, I hand-roll my own new-page creation. I could also hook my logic into Org-Roam’s templating system Details: https://github.com/org-roam/org-roam/blob/c3867619147175faf89ed8f3e90a1e67a4fd9655/doc/org-roam.org#L896

2022-08-20 I’m happy with how it’s working now.

Solution: write my own Emacs lisp library to control page creation and page search.

Really happy. I consider it solved now.

comment system

Option: Comments on Twitter. Option: Comments on Hacker News. Option: Comments embedded straight into the text - via Github pull requests.

Option: Comments as Github issues. Example: https://github.com/matthijscox/Blog/issues/3

2022-08-20 Just share on relevant Slack, where there can be comments

Comment system is the sharing mechanism. Don’t expect to find too many people “just cause”.

Date tagging

Problem statement — should I encode dates (created, edited) in the metadata model?

Early view

I want metadata for :changed and :created. First because I want to filter / sort. Then perhaps later because that metadata is nice to have.

2022-07-12 says pikaboo

Date tagging would be nice. But by what semantics? And why?

Reasonable question — which vague idea was created when? Which vague idea was edited last? I could use OS mtime for editing. Sync into play.edn. Have play.edn files partially generated. Or just pull it out from source at the right time? Do the :relations trick?

  1. I really like the idea of just using file modification times.

    buuut I don’t think those get checked into git

  2. Can I just use git blame perhaps?

    🤔

    Git knows when the file was changed last

  3. Outline — on page creation + git blame

    On creating a new page, write :created (now)

    On creating the metadata table, file edit time by asking Git

2022-07-14 👀

Well — haven’t really taken this any further.

I kinda want dates for my ephemeral stuff. So that it’s easier to handle the very incomplete stuff. Or at least sort by creation date.

Hmm, let’s just add a :created timestamp.

I want to derive :last-modified from git history of org-file.

2022-05-26

From #teodor-discuss (Iterate slack)

Source: https://garasjen.slack.com/archives/C01KB5RSAU9/p1653562307600869

Hva vil jeg egentlig å få ut av en nettside?

Effektivt å skrive tekst. Kan gjøre det i vanlige tekstformater. Frittstående sider. Løs kobling, trenger ikke samme struktur overalt. Gjenbrukbare komponenter. Hele greia funker som en statisk side. Statisk html er sjekket inn i git. Lynraske bygg. Effektivt å jobbe med hiccup.

Effektivt å jobbe med ideer. Effektivt å lenke mellom ideer.

Lett å få inn ting jeg har tegnet. Feks på papir, remarkable eller i figma

Twitter… Trenger jeg å tenke på det? Kommentarer. Webmentions? Systemet jeg bruker i dag? Hacker news? Twitter? Jeg ønsker jo å kunne diskutere ting på Twitter Og det gir mening å ha invitasjoner til diskusjon på Twitter, samt invitasjon til å diskutere på Twitter på nettsiden.

Hva med unicad og blogging og artikler for sånt?

Approach: embed unicad I noe annet. Approach: bygg skriving rett inn i unicad.

Hva får jeg egentlig ut av en nettside som jeg ikke får her i #teodor-discuss? Tilgjengelig for folk utenfra Bedre på å redigere store ting Mer kontroll på presentasjon (også mer jobb)

Oddmund: Har du vurdert å bare skrive nettsida di i HTML?

Det er jo cirka det jeg gjør på subcons.teod.eu. Bare via hiccup

Kan laste opp rå html også, og det funker fint.

Ting jeg ønsker å skrive om: Ortogonalitet Feedback-loop + API-design + hvordan ting funker Eksempler på bruk av watchexec Noe dataviz

@oddmunds jeg har prøvd å skrive html direkte, men det føles som å gå i gjørme. Med hiccup har jeg i alle fall en skikkelig editor. Men jeg liker bedre å redigere tekst i gode formater for tekst (feks org-mode, eller markdown, til en viss grad)

Enda en side: Konvertering mellom Roam / html / hiccup / org-mode Litt usikker på om jeg får til det rett i browser. Trenger kanskje en tjeneste. En backend. Med pandoc og Clojure bør det ikke være alt for vanskelig. Men hvis jeg får opp noe sånt, kan jeg lett gå mellom formater.

Enda en side: Signaler, starting av prosesser, stopping av prosesser. Sigterm, sigkill, sighup (?) Kan lage en side i stedet for å lage presentasjon eller noe annet.

Why? I like creating sections for discussion and changelog in the end of an article. But I don’t want those to pollute the table of contents! So I need to take control of that.

Specific example - for “Strong opinions loosely held” is an excuse for sloppy thinking, my TOC as of 2022-12-19 is:

Part 1 – a taxonomy of knowledge. Part 2 – so what? Part 3 – write shit down. Discussion 2022-12-17 . Changelog 2022-12-19 .

I don’t want “Discussion”, “Changelog” and “.” in the TOC!

2023-02-14

new pandoc, new style

pandoc --version

Today is the first time I’m writing (again) after getting Pandoc 3. And I’m really appreciating the changes! It’s cleaner, and takes up less space.

2023-03-07

I was motivated to write some stuff today.

what pages did I create today?

Principle of Charity Learning to discover Support, challenge, carry

Nice!

It would be nice to list all pages by creation time.

I could solve that with an index query, and present it on a page.

I’d like batch tag editing.

Options:

  1. Manual editing
  2. Emacs UI
  3. CLI
  4. Electric Clojure UI
  5. Functionality in a lib, usable from CLI and electric Clojure

In general, I want a batch editing UI

Emacs could be nice, but I’d rather want an Electric Clojure UI, I think. Just run it locally for now. 🤔

2023-03-09

IKI UI

I want an IKI UI. I think browser UI is the way to go.

Proposed functionality

  1. List recent pages
  2. Edit titles
  3. Batch edit page tags
    1. Perhaps first class actions should be a thing? Named actions?

Why?

in order to increase trust, shared sense of quality and shared intent with my network.

2023-03-20

Simple CSS?

inspiration: https://markmcgranaghan.com/ and https://gobyexample.com/ by Mark McGranaghan.

2023-08-27

Jacobian’s site has nice CSS

https://jacobian.org/2023/mar/31/incompetent-but-nice-follow-ups/

I especially like his quotation style. It’s light, just enough.

2024-01-21

Problem: not validating slugs

Observation:

  1. I just created a mess for myself by creating a page with a slug with spaces in it.
  2. The result was an error state, I had to manually delete files

Proposed solution 1:

  1. Create validation in the page create script that prevents this from ever happening!

🐛 build is broken

I can’t build! Let’s fix it.

how to reproduce

Version info:

$ git rev-parse HEAD
0c27fa27b1b824670e6c994e6aee972c87c54dd7
$ pandoc --version
pandoc 3.1.11.1
Features: +server +lua
Scripting engine: Lua 5.4
User data directory: /Users/teodorlu/.local/share/pandoc
Copyright (C) 2006-2023 John MacFarlane. Web: https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.
$ bb --version
babashka v1.3.188
$ make
pandoc -s --shift-heading-level-by=1 --from=org+smart -i compressing-thoughts/index.org -t json | ./play.clj filter resolve-links | pandoc -f json -o compressing-thoughts/index.html --standalone --toc -H header-default-include.html -H scittle/scittle-with-extras.html
JSON parse error: Error in $: not enough input
make: *** [compressing-thoughts/index.html] Error 64

Is the hypothesis correct?

$ echo '# hei duuu' | pandoc --from markdown --to json
{"pandoc-api-version":[1,23,1],"meta":{},"blocks":[{"t":"Header","c":[1,["hei-duuu",[],[]],[{"t":"Str","c":"hei"},{"t":"Space"},{"t":"Str","c":"duuu"}]]}]}
$ echo '# hei duuu' | pandoc --from markdown --to json | ./play.clj filter resolve-links

No output from ./play.clj filter resolve-links. That’s a problem.

??

  1. does anyting run?

    Let’s see what happens.

    $ git diff src
    diff --git a/src/teod/play/cli.clj b/src/teod/play/cli.clj
    index 6da93c6f..f203b38c 100644
    --- a/src/teod/play/cli.clj
    +++ b/src/teod/play/cli.clj
    @@ -395,7 +395,12 @@ Allowed options:
             (spit "index/big.json" (json/generate-string @big-index {:pretty true}))
             ))))
    
    +(defn verbose? []
    +  (not (nil? (System/getenv "EU_TEOD_PLAY_VERBOSE"))))
    +
     (defn filter [{:as cmd-opts}]
    +  (when (verbose?) (prn "it runs"))
    +
       ;; only supported filter for now is resolve-links
       ;;
       ;; Test with:
    

    then

    $ export EU_TEOD_PLAY_VERBOSE=1
    $ echo '# hei duuu' | pandoc --from markdown --to json | ./play.clj filter resolve-links
    "it runs"
    

    something runs.

  2. hypothesis: we branch away from printing (code bug)

    $ git diff src
    diff --git a/src/teod/play/cli.clj b/src/teod/play/cli.clj
    index f203b38c..0a4424e9 100644
    --- a/src/teod/play/cli.clj
    +++ b/src/teod/play/cli.clj
    @@ -417,6 +417,11 @@ Usage:
    
       (when (contains? (set (:rest-cmds cmd-opts))
                        "resolve-links")
    +    (when (verbose?)
    +      (prn
    +       '(:rest-cmds cmd-opts)
    +       'contains?
    +       "resolve-links"))
         (let [pandoc-json (json/parse-string (slurp *in*))
               by-uuid (fn [uuid]
                         (let [path (str "index/by-uuid/" uuid ".edn")]
    

    then

    $ echo '# hei duuu' | pandoc --from markdown --to json | ./play.clj filter resolve-links
    "it runs"
    

    !!! We we have our problem.

:rest-cmds is deprecated and will be removed

From https://github.com/babashka/cli/blob/8ec3e9b8e60b1d37f2377b86c5c448f3938a0b7d/src/babashka/cli.cljc#L685


(defn dispatch
  "Subcommand dispatcher.

  Dispatches on longest matching command entry in `table` by matching
  subcommands to the `:cmds` vector and invoking the correspondig `:fn`.

  Table is in the form:

  ```clojure
  [{:cmds [\"sub_1\" .. \"sub_n\"] :fn f :cmds-opts [:lib]}
   ...
   {:cmds [] :fn f}]
  ```

  When a match is found, `:fn` called with the return value of
  `parse-args` applied to `args` enhanced with:

  * `:dispatch` - the matching commands
  * `:args` - concatenation of unparsed commands and args
  * `:rest-cmds`: DEPRECATED, this will be removed in a future version

  Use an empty `:cmds` vector to always match or to provide global options.

  Provide an `:error-fn` to deal with non-matches.

  Each entry in the table may have additional `parse-args` options.

  For more information and examples, see [README.md](README.md#subcommands)."
  ([table args]
   (dispatch table args {}))
  ([table args opts]
   (let [tree (-> table table->tree)]
     (dispatch-tree tree args opts))))

I’ll use :args instead.

New code:

https://github.com/teodorlu/play.teod.eu/blob/68007b1143c691ba3cf49057ef91e221e0510994/src/teod/play/cli.clj#L419-L453

2024-03-29

I want better page load performance

Now:

  1. Go to random page button is implemented as runtime interpreted ClojureScript with Scittle, and react. Multiple dependencies that load in a weird order.
  2. Each xx
  3. On each page load, I re-download the index of all the pages.

Plan:

  1. Use JavaScript modules to download page index once
  2. Write logic in plain javascript

I want a nice way to embed Clerk notebooks

I want to keep on using Pandoc to parse org-mode, but I want my own HTML converter

Why?

  1. There’s a Pandoc bug where the TOC is wrong when I override an Org-mode element ID
  2. I want to learn CSS
  3. I want to fix table of contents visibility issues.

2024-07-11

Multiple interfaces

Per 2024-07-11, I have three ways of doing things on play.teod.eu:

  1. Via the Emacs interface
  2. Via the CLI
  3. Via the Clojure interface

Multiple interfaces is nice, but subtle differences in the three interfaces is not so nice. Namely:

  1. I need to maintain different mental models about all three interfaces
  2. Differences between the three interfaces make it harder to get things done without introducing new problems

Why? I feel a certain friction when I want to change things now. I lean on the fact that it “works” as it is now.