Databasekøar, batching og 1400 GB XML

Dette innlegget vart fyrst publisert på kode24.no. Bakgrunn Eg hadde sagt ja til å forbetra hastigheita på eit legacysystem. Kor vanskeleg kunne det vera? Eg hadde fått det for meg at det var snakk om rundt 70 GB XML det skulle prosessera. Det viste seg å vera 1400 GB XML. Kjeldekoden var skriven i Python 2 (end of life i 2020), hadde ingen testar, ingen spesifiserte avhengigheiter og innehaldt i tillegg rundt ti tusen linjer med SQL, primært i form av prosedyrer. ...

2025-08-08 · Ivar Refsdal

Ctrl-S is my REPL: GitLab

Introduction Sometimes I need to fiddle with .gitlab-ci.yml. What variables are set? Will this runner image have the available tools I need? Did I forget a ; inside some multiline bash string of sorts? Who doesn’t enjoy occasionally writing bash scripts in YAML? This would warrant a fast feedback loop. Until recently my “loop” has been like this: Ctrl-S (or Cmd-S as I’m on OS X). Alt-0 (git changes) Space (mark file to commit) Alt-P

2024-08-12 · Ivar Refsdal

Datomic's handling of network failures

Introduction This post will examine how the Datomic on-premise peer library handles and responds to network failures. The version tested is 1.0.7075, released 2023-12-18. Datomic uses the Apache Tomcat JDBC Connection Pool for SQL connection management. PostgreSQL is used as the underlying storage in this test. More specifically we will be testing: com.datomic/peer 1.0.7075 ; Released 2023-12-18 org.apache.tomcat/tomcat-jdbc 7.0.109 (bundled by datomic-pro) org.postgresql/postgresql 42.5.1 OpenJDK 64-Bit Server VM Temurin-22+36 (build 22+36, mixed mode, sharing) We will be using nftables to simulate network errors. ...

2024-05-19 · Ivar Refsdal