ob-elixir/docs/05-existing-implementations-analysis.md

Existing Implementations Analysis

This document analyzes existing org-babel Elixir implementations and related packages, identifying gaps and improvement opportunities.

Table of Contents

• zweifisch/ob-elixir Analysis
• Comparison with Other ob-* Packages
• Feature Gap Analysis
• Recommendations
• References

---

zweifisch/ob-elixir Analysis

Repository Information

• URL: https://github.com/zweifisch/ob-elixir
• Stars: 29 (as of 2024)
• Last Updated: Limited recent activity
• License: GPL-3.0

Source Code Review

;;; Current implementation (simplified)

(defvar ob-elixir-process-output "")

(defconst org-babel-header-args:elixir
  '((cookie . :any)
    (name . :any)
    (remsh . :any)
    (sname . :any))
  "elixir header arguments")

(defvar ob-elixir-eoe "\u2029")  ; Unicode paragraph separator

(defun org-babel-execute:elixir (body params)
  (let ((session (cdr (assoc :session params)))
        (tmp (org-babel-temp-file "elixir-")))
    (ob-elixir-ensure-session session params)
    (with-temp-file tmp (insert body))
    (ob-elixir-eval session (format "import_file(\"%s\")" tmp))))

(defun ob-elixir-eval (session body)
  (let ((result (ob-elixir-eval-in-repl session body)))
    ;; Heavy regex cleanup of IEx output
    (replace-regexp-in-string
     "^\\(import_file([^)]+)\\)+\n" ""
     (replace-regexp-in-string
      "\r" ""
      (replace-regexp-in-string
       "\n\\(\\(iex\\|[.]+\\)\\(([^@]+@[^)]+)[0-9]+\\|([0-9]+)\\)> \\)+" ""
       (replace-regexp-in-string
        "\e\\[[0-9;]*[A-Za-z]" ""
        (replace-regexp-in-string
         "\"\\\\u2029\"" ""
         result)))))))

Strengths

Feature	Description
Session support	Uses IEx sessions for persistent state
Remote shell	Supports --remsh for connecting to running nodes
Node naming	Supports --sname and --name for distributed Erlang
Cookie support	Can specify Erlang cookie for authentication
Simple design	Minimal code, easy to understand

Weaknesses

Issue	Description	Impact
Always uses sessions	No one-shot execution mode	Slow for simple scripts
No :results value	Only captures output, not return values	Limited functionality
No :var support	Cannot pass variables to code blocks	Poor org-babel integration
Fragile output parsing	Multiple regex replacements to clean IEx output	Unreliable
Uses import_file	Relies on IEx-specific command	Tight coupling to IEx
Global state	Uses ob-elixir-process-output global variable	Not thread-safe
No Mix support	Cannot execute in Mix project context	Limited for real projects
Hardcoded sit-for	Uses fixed delays instead of proper synchronization	Timing issues
No error handling	Errors not properly detected or reported	Poor UX
No tests	No test suite	Quality concerns
No defcustom	Settings not customizable via Customize	Poor UX

Specific Issues in Code

1. Timing-based synchronization:

(sit-for 0.5)  ; Wait 500ms and hope IEx is ready
;; This is fragile - slow systems may fail

2. No prompt detection:

(defun ob-elixir-wait ()
  (while (not (string-match-p ob-elixir-eoe ob-elixir-process-output))
    (sit-for 0.2)))
;; Busy-waiting instead of using process filter properly

3. Heavy output cleanup:

;; 5 nested regex replacements - fragile and hard to debug
(replace-regexp-in-string "pattern1" ""
  (replace-regexp-in-string "pattern2" ""
    (replace-regexp-in-string "pattern3" ""
      ...)))

---

Comparison with Other ob-* Packages

ob-python (Standard Library)

Features we should match:

;; Multiple execution modes
(defcustom org-babel-python-command "python3"
  "Command to execute Python code.")

;; Proper result handling
(defun org-babel-execute:python (body params)
  (let* ((session (org-babel-python-initiate-session
                   (cdr (assq :session params)) params))
         (result-params (cdr (assq :result-params params)))
         (result-type (cdr (assq :result-type params)))
         (full-body (org-babel-expand-body:generic ...))
         (result (org-babel-python-evaluate ...)))
    ;; Proper result formatting
    (org-babel-reassemble-table
     (org-babel-result-cond result-params
       result
       (org-babel-python-table-or-string result))
     ...)))

;; Variable injection
(defun org-babel-variable-assignments:python (params)
  (mapcar (lambda (pair)
            (format "%s=%s" (car pair)
                    (org-babel-python-var-to-python (cdr pair))))
          (org-babel--get-vars params)))

;; Both session and non-session evaluation
(defun org-babel-python-evaluate (session body &optional result-type result-params)
  (if session
      (org-babel-python-evaluate-session ...)
    (org-babel-python-evaluate-external-process ...)))

ob-ruby (Standard Library)

Patterns to adopt:

;; Clean wrapper for value capture
(defconst org-babel-ruby-wrapper-method
  "
def main
%s
end
puts main.inspect
")

;; Proper session management with comint
(defun org-babel-ruby-initiate-session (&optional session params)
  (unless (string= session "none")
    (let ((session-buffer (or (get-buffer ...) ...)))
      (if (org-babel-comint-buffer-livep session-buffer)
          session-buffer
        (org-babel-ruby-initiate-session session)))))

;; Type conversion
(defun org-babel-ruby-var-to-ruby (var)
  (if (listp var)
      (concat "[" (mapconcat #'org-babel-ruby-var-to-ruby var ", ") "]")
    (format "%S" var)))

ob-shell (Standard Library)

Useful patterns:

;; Multiple shell types
(defcustom org-babel-shell-names
  '("sh" "bash" "zsh" "fish" ...)
  "Shells to register with org-babel.")

;; Header arg for shell selection
(defconst org-babel-header-args:shell
  '((shebang . :any)))

;; Proper prep-session
(defun org-babel-prep-session:shell (session params)
  (let* ((session (org-babel-sh-initiate-session session))
         (var-lines (org-babel-variable-assignments:shell params)))
    (org-babel-comint-in-buffer session
      (mapc (lambda (var)
              (insert var) (comint-send-input nil t)
              (org-babel-comint-wait-for-output session))
            var-lines))
    session))

Feature Comparison Matrix

Feature	ob-python	ob-ruby	ob-shell	ob-elixir (zweifisch)
One-shot execution	Yes	Yes	Yes	No
Session support	Yes	Yes	Yes	Yes
:results value	Yes	Yes	Yes	No
:results output	Yes	Yes	Yes	Yes (only)
Variable injection	Yes	Yes	Yes	No
Table output	Yes	Yes	Yes	No
Error handling	Yes	Yes	Yes	No
Customization	Yes	Yes	Yes	No
Tests	Yes	Yes	Yes	No
Graphics support	Yes	No	No	No
Async support	Yes	No	No	No

---

Feature Gap Analysis

Critical Gaps (Must Fix)

1. No :results value support

   • Cannot capture return value of expressions
   • Users must use IO.inspect workarounds
   • Fix: Implement wrapper method pattern

2. No variable injection

   • Cannot use :var header argument
   • Breaks org-babel's data passing paradigm
   • Fix: Implement org-babel-variable-assignments:elixir

3. No one-shot execution

   • Every evaluation starts/uses IEx session
   • Slow for simple scripts
   • Fix: Add external process execution mode

4. Fragile output parsing

   • Multiple regex replacements prone to breaking
   • Fix: Use markers or JSON encoding

Important Gaps (Should Fix)

1. No Mix project support

   • Cannot run code in project context
   • No access to project dependencies
   • Fix: Add :mix-project header argument

2. No proper error handling

   • Errors appear as raw output
   • No structured error information
   • Fix: Detect and signal Elixir errors

3. No customization

   • Hardcoded paths and settings
   • Fix: Add defcustom for all configurable values

4. No tests

   • No way to verify correctness
   • Fix: Add comprehensive test suite

Nice-to-Have Gaps (Could Fix)

1. No async execution

   • Long-running code blocks UI
   • Consider for future versions

2. No graphics support

   • Elixir has visualization libraries (VegaLite)
   • Could add in future

3. No LSP integration

   • Could integrate with ElixirLS for completions
   • Future enhancement

---

Recommendations

Approach: New Implementation

Given the significant gaps and architectural issues in zweifisch/ob-elixir, we recommend creating a new implementation rather than forking. Reasons:

1. Need to change fundamental architecture (session-only → dual mode)
2. Core data flow needs redesign
3. Limited active maintenance upstream
4. Cleaner API design possible with fresh start

Proposed Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     org-babel-execute:elixir                     │
└─────────────────────────────────────────────────────────────────┘
                               │
                    ┌──────────┴──────────┐
                    │                     │
                    ▼                     ▼
          ┌─────────────────┐   ┌─────────────────┐
          │ Session Mode    │   │ External Mode   │
          │ (IEx/comint)    │   │ (elixir cmd)    │
          └─────────────────┘   └─────────────────┘
                    │                     │
                    ▼                     ▼
          ┌─────────────────┐   ┌─────────────────┐
          │ With Mix        │   │ With Mix        │
          │ (iex -S mix)    │   │ (mix run)       │
          └─────────────────┘   └─────────────────┘

Implementation Phases

Phase 1: Core (MVP)

• One-shot execution with elixir command
• :results value and :results output support
• Variable injection with :var
• Basic error detection
• Customizable commands
• Test suite foundation

Phase 2: Sessions

• IEx session support via comint
• Session persistence
• Proper prompt detection
• Session cleanup

Phase 3: Mix Integration

• :mix-project header argument
• Automatic Mix project detection
• iex -S mix for sessions
• mix run for one-shot

Phase 4: Advanced Features

• Remote shell (remsh) support
• Table input/output
• Async execution
• Better error messages with line numbers

Header Arguments to Support

(defconst org-babel-header-args:elixir
  '(;; Standard org-babel args work automatically
    ;; Language-specific:
    (mix-project . :any)      ; Path to mix.exs directory
    (mix-env     . :any)      ; MIX_ENV (dev, test, prod)
    (iex-args    . :any)      ; Extra IEx arguments
    (timeout     . :any)      ; Execution timeout
    (node-name   . :any)      ; --name for distributed
    (node-sname  . :any)      ; --sname for distributed
    (cookie      . :any)      ; Erlang cookie
    (remsh       . :any))     ; Remote shell node
  "Elixir-specific header arguments.")

Example Usage (Target)

* Basic Evaluation

#+BEGIN_SRC elixir
Enum.map([1, 2, 3], &(&1 * 2))
#+END_SRC

#+RESULTS:
| 2 | 4 | 6 |

* With Variables

#+NAME: my-data
| a | 1 |
| b | 2 |

#+BEGIN_SRC elixir :var data=my-data
Enum.map(data, fn [k, v] -> {k, v * 10} end)
#+END_SRC

#+RESULTS:
| a | 10 |
| b | 20 |

* In Mix Project

#+BEGIN_SRC elixir :mix-project ~/my_app :mix-env test
MyApp.some_function()
#+END_SRC

* With Session

#+BEGIN_SRC elixir :session my-session
defmodule Helper do
  def double(x), do: x * 2
end
#+END_SRC

#+BEGIN_SRC elixir :session my-session
Helper.double(21)
#+END_SRC

#+RESULTS:
: 42

* Capturing Output

#+BEGIN_SRC elixir :results output
IO.puts("Hello")
IO.puts("World")
#+END_SRC

#+RESULTS:
: Hello
: World

---

References

• zweifisch/ob-elixir
• Org Mode ob-python.el
• Org Mode ob-ruby.el
• Org Mode ob-shell.el
• Org Mode ob-emacs-lisp.el
• elixir-editors/emacs-elixir
• Worg Babel Languages