In programming, names should make it clear whether you’re talking about a category of things or a specific entity. I commonly see programmers work with entities, but name the category. The name is too wide, if we use Zach Tellman’s terminology from Elements of Clojure.
I’m still not used to playing around with stateful systems. I avoid state like the plague! Should I change that? Perhaps?
If yes, how? Write an interactive chat application? Write some logging thing? A way to visualize those logs?
Hmmm.
Since last “State pling plong”, I set up a PostgreSQL database for mikrobloggeriet.no. That was a helpful learning experience. I’m glad i spent the time.
https://github.com/igrishaev/pg2
https://github.com/igrishaev/pg2/issues/1
When I tried starting a REPL like I normally do, it didn’t work.
lots of changes.
README.md | 297 +++++++++++++++++++++++++++++++++++++-
pg-bench/src/pg/demo.clj | 197 ++++++++++++++++++++++++-
pg-core/src/clj/pg/core.clj | 60 +++++---
pg-core/src/clj/pg/jdbc.clj | 262 +++++++++++++++++++++++++++++++++
pg-core/src/clj/pg/pool.clj | 10 +-
pg-core/src/java/org/pg/Accum.java | 4 +-
pg-core/src/java/org/pg/ConnConfig.java | 19 ++-
pg-core/src/java/org/pg/Connection.java | 21 ++-
pg-core/src/java/org/pg/ExecuteParams.java | 1 +
pg-core/src/java/org/pg/Main.java | 9 +-
pg-core/src/java/org/pg/Payload.java | 1 +
pg-core/src/java/org/pg/auth/ScramSha256.java | 2 +-
pg-core/src/java/org/pg/codec/DecoderBin.java | 2 +-
pg-core/src/java/org/pg/codec/DecoderTxt.java | 2 +-
pg-core/src/java/org/pg/codec/EncoderBin.java | 5 +-
pg-core/src/java/org/pg/codec/EncoderTxt.java | 18 ++-
pg-core/src/java/org/pg/enums/Format.java | 2 +-
pg-core/src/java/org/pg/enums/SASL.java | 2 +-
pg-core/src/java/org/pg/enums/TXStatus.java | 2 +-
pg-core/src/java/org/pg/{ => error}/PGError.java | 2 +-
pg-core/src/java/org/pg/{ => error}/PGErrorResponse.java | 2 +-
pg-core/src/java/org/pg/json/JSON.java | 2 +-
pg-core/src/java/org/pg/msg/AuthenticationResponse.java | 2 +-
pg-core/src/java/org/pg/msg/ErrorResponse.java | 2 +-
pg-core/src/java/org/pg/msg/Execute.java | 2 +-
pg-core/src/java/org/pg/msg/FieldParser.java | 2 +-
pg-core/src/java/org/pg/msg/NegotiateProtocolVersion.java | 2 +-
pg-core/src/java/org/pg/msg/NoticeResponse.java | 2 +-
pg-core/src/java/org/pg/msg/NotificationResponse.java | 2 +-
pg-core/src/java/org/pg/msg/Parse.java | 2 +-
pg-core/src/java/org/pg/pool/Pool.java | 22 ++-
pg-core/src/java/org/pg/{clojure => proto}/IClojure.java | 2 +-
pg-core/src/java/org/pg/{reducer => proto}/IReducer.java | 2 +-
pg-core/src/java/org/pg/reducer/Default.java | 1 +
pg-core/src/java/org/pg/reducer/Dummy.java | 1 +
pg-core/src/java/org/pg/reducer/First.java | 1 +
pg-core/src/java/org/pg/reducer/Fold.java | 1 +
pg-core/src/java/org/pg/reducer/GroupBy.java | 1 +
pg-core/src/java/org/pg/reducer/IndexBy.java | 1 +
pg-core/src/java/org/pg/reducer/Java.java | 1 +
pg-core/src/java/org/pg/reducer/KV.java | 1 +
pg-core/src/java/org/pg/reducer/Matrix.java | 1 +
pg-core/src/java/org/pg/reducer/Run.java | 1 +
pg-core/src/java/org/pg/type/PGEnum.java | 2 +-
pg-core/src/java/org/pg/util/HashTool.java | 2 +-
pg-core/src/java/org/pg/util/HexTool.java | 2 +-
pg-core/src/java/org/pg/util/IOTool.java | 2 +-
pg-core/test/pg/client_test.clj | 18 ++-
pg-core/test/pg/concurrency_test.clj | 164 +++++++++++++++++++--
pg-core/test/pg/decode_txt_test.clj | 2 +-
pg-core/test/pg/encode_bin_test.clj | 2 +-
pg-core/test/pg/encode_txt_test.clj | 2 +-
pg-core/test/pg/jdbc_test.clj | 289 +++++++++++++++++++++++++++++++++++++
pg-core/test/pg/pool_test.clj | 4 +-
todo.md | 26 +++-
README, demo, tests. But “jdbc”??? Why jdbc?
(ns pg.jdbc
"
The Next.JDBC-friendly wrapper. Mimics most of the
`next.jdbc` functions and macros.
https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc.clj
"
(:import
clojure.lang.IPersistentMap
clojure.lang.IFn
org.pg.Connection
org.pg.pool.Pool)
(:require
[clojure.set :as set]
[pg.core :as pg]
[pg.pool :as pool]))
specifically
The Next.JDBC-friendly wrapper. Mimics most of the `next.jdbc` functions and macros.
Yeeeeah, OK. Not “let’s use jdbc and compare to pg2 to test our code”, but “let’s provide a nice comptibility layer for those using next.jdbc today.”
Let’s see.
docker-compose.yaml with lots of
different pg versions.project.clj contains :managed-versions and :sub
:managed-versions: ??:java-source-paths: hei.
Jeg har nå gjort cirka følgende fra toppen:
(cd pg-core && lein clean && lein javac && lein install)
(cd pg-honey && lein clean && lein javac && lein install)
(cd pg-component && lein clean && lein javac && lein install)
… og jeg får nå andre typer feil når jeg prøver å kjøre
testene (feks med (cd pg-core && lein test)).
docker compose upFeilene jeg fikk over så ut som “prøver å koble til port, finner ikke”
lein test pg.client-test
lein test :only pg.client-test/test-decode-oid
ERROR in (test-decode-oid) (IOTool.java:139)
PORT 10110
Uncaught exception, not in assertion.
expected: nil
actual: org.pg.PGError: cannot open a socket, host: 127.0.0.1, port: 10110
at org.pg.util.IOTool.socket (IOTool.java:139)
org.pg.Connection._connect_unlocked (Connection.java:304)
org.pg.Connection.connect (Connection.java:297)
org.pg.Connection.<init> (Connection.java:73)
org.pg.Connection.<init> (Connection.java:82)
[...]
Må jeg kjøre opp en PostgreSQL-instans lokalt?
$ docker compose up
[...]
[+] Running 1/0
✔ Network pg2_default Created 0.0s
⠋ Container pg2-pg14-1 Creating 0.0s
⠋ Container pg2-pg11-1 Creating 0.0s
⠋ Container pg2-pg15-1 Creating 0.0s
⠋ Container pg2-pg12-1 Creating 0.0s
⠋ Container pg2-pg16-1 Creating 0.0s
⠋ Container pg2-pg13-1 Creating 0.0s
Error response from daemon: make cli opts(): making volume mountpoint for volume /Users/teodorlu/dev/igrishaev/pg2/.docker/postgres/initdb.d: mkdir /Users: operation not permitted
Error: executing /opt/homebrew/bin/docker-compose up: exit status 1
Hmmmm.
Heh, det finnes jo makefiles overalt! :D De må jeg prøve.
daniel stuff
using ggplot from clojure work by generateme - how to use ggplot from Clojure (clojisr)
90 minutes total
Teodor
Programming, civil engineering, product. Interested in data and data visualization, not an expert.
Data Visualization: not an expert, and would definitely like to get better.
thomas 2018
frontend databases, dashboards, visuals get into the data part “r is a great place to start”
d3, three.js “grammer of graphics”
wrote a gallery
daniel
community organizer scicloj funded by clojurists together
andrew
dataviz and data science python, scala, r, clojure prior experience with ggplot
found ggplot to be superior for fundamental reasons primarily clojure now think there is a lot to learn from ggplot
implement wilkinsons “grammar of graphics”
“there’s a reason why ggplot2 is THE dataviz library for R” explore more and learn.
daniel: what is grammar of graphics
kira
possible to do dataviz with vega / vega-lite “dataviz biggest hole”
really cool to implement the grammar of graphics in clojure it’s an idea, it’s a grammar
“right now, only impl is in R”
tomasz
written clojure libraries created “cljplot” from scratch, “lots of mistakes”, hope to contribute
jaryt
phd student, small university midwest taught at a small university in africa here to study, “be with the best!”
intro, andrew
in the clojure world, we work with vega & vega lite impl of grammar of graphics
from leol wilkinson early work 80s statistical computing data visualization access to computers was new spss “chief scientist” “h2o AI” passed away grammar of graphics is his “magnus opus”
important because — page 2 —
first part, syntax
we often call graphics charts (greek), latin pie charts, bar charts
shuns charts
charts instances of general objects
pie is a divided bar into polar coordinates
histogram != bar charts
different grammars
fewer charts than wanted no deep structure no reuse cannot add new charts “theory of graphics, not charts”
decompose basic charts into the more fundamental things axes, scales coordinates
ways of combining formal algebra
r library ggplot “much more directly implements the algebra” than vega (lite) hypothesis: we’ll see ggplot2 perform better than vega (lite).
Wilkinson book: https://link.springer.com/book/10.1007/0-387-28695-0
Other book: https://ggplot2-book.org/
daniel demo
Qs teodor
(r "1 + 1").
ggplot vs vega
ggplot has a combinator API
ways to combine primitives!
kira: about vega-lite, “have to understand everything in order to do the fancy stuff” axes, layers, scales don’t want to care marks, encodings have that magic “behind the scenes” translation between high-level easy to EDN/JSON/…
andrew: “incremental specificity”, “compositional power”
daniel: functions, objects, “hidden” teodor: emmy, reitit, pure data representation is opt-in
daniel: jun cho wants to look inside, ggplot is hiding ggplot-like “functions and safety”, can check if args are ok perhaps we end up with something safe to hanami
daniel demo
teodor: what if we start by translating from r
daniel: “stat layer” in your dataflow
“hard”. Actionabes?
scale: log scale, linear scale grammar: roles of “things” in function calls
nice book.
chapter 13 - build a plot layer by layer https://ggplot2-book.org/layers
thomasz has a nice backend, but is looking for a better way to create the plots.
kira: move processing to the JVM side. CLJplot: crunching in Clojure on the JVM
andrew: humble UI
thomas: learn, document the learning! kira: learn, document the learning!
daniel: look into hanami. Defaults and some logic around defaults as data.
Teodor: separate speed from design.
kira: sensible defaults, meta-merge them up current impl: proliferation of chart templates, infinite number of possible chart specifications. general data science work: need more general things.
“what’s better, vega, hanami or ggplot?” Kira: you don’t have to understand everything in ggplot.
goal: can make it work without understanding what’s under the abstractions goal: make it terse
teodor ggplot draft
(ns explore.ggplotj
(:refer-clojure :exclude [+]))
;; STEP 1: OPERATIONS AND EXAMPLE USAGE
(defn + [])
(defn ggplot [])
(defn aes [])
(defn geom-point [])
;; STEP 2: PROTOCOLS AND STRUCTS
;;
;; ... assuming we don't need two-argument dispatch.
(comment
+
ggplot
aes
geom-point)alternative name: jjplot. Symmetry
& hat tip to ggplot. Hat tip to CloJure & Jvm.
Mostly because unpgk is slow to return.
Lessons:
Decision: disable livejs temporarily, rerun bench.
I’ve made play.teod.eu with a goal in mind:
Intent: bring ideas to life. Discuss, sharpen, play. Minimize the distance between intent and reality.
Without that goal, I would have worked differently. That goal helps me know how to balance the amount of technical debt in my system. Rather than code to code, I code to achieve some outside goal!
Lately, I discovered that I had 360 lower-case formatted UUIDs in my system, and 36 upper-case formatted UUIDs. See for instance commit 8f62a0. This is kind of embarrassing! But, what do I do with it? Fix it?
I decided on the following:
Stumbled over Eva Parish and her text What I think about when I edit, which I enjoyed a lot.
About a skill I value, a skill where I have something to learn, and a skill I wish more people would value.
| subjektiv | objektiv | |
| liten | mening | observasjon |
| stor | vurdering | slutning |
| subjective | objective | |
| small | opinion | observation |
| big | deduction | judgment |
decided to put this table in a tweet: https://twitter.com/TeodorHeggelund/status/1779112730001125852
opinion ≠ observation, deduction ≠ judgment
I think we could avoid a lot of pain if we put in a bit more effort to separate between “objective falsifiable” (let’s find the facts!) and “subjective” (let’s explore taste!).