436 lines
14 KiB
Markdown
436 lines
14 KiB
Markdown
# Existing Implementations Analysis
|
|
|
|
This document analyzes existing org-babel Elixir implementations and related packages, identifying gaps and improvement opportunities.
|
|
|
|
## Table of Contents
|
|
|
|
- [zweifisch/ob-elixir Analysis](#zweifischob-elixir-analysis)
|
|
- [Comparison with Other ob-* Packages](#comparison-with-other-ob--packages)
|
|
- [Feature Gap Analysis](#feature-gap-analysis)
|
|
- [Recommendations](#recommendations)
|
|
- [References](#references)
|
|
|
|
---
|
|
|
|
## zweifisch/ob-elixir Analysis
|
|
|
|
### Repository Information
|
|
|
|
- **URL**: https://github.com/zweifisch/ob-elixir
|
|
- **Stars**: 29 (as of 2024)
|
|
- **Last Updated**: Limited recent activity
|
|
- **License**: GPL-3.0
|
|
|
|
### Source Code Review
|
|
|
|
```elisp
|
|
;;; Current implementation (simplified)
|
|
|
|
(defvar ob-elixir-process-output "")
|
|
|
|
(defconst org-babel-header-args:elixir
|
|
'((cookie . :any)
|
|
(name . :any)
|
|
(remsh . :any)
|
|
(sname . :any))
|
|
"elixir header arguments")
|
|
|
|
(defvar ob-elixir-eoe "\u2029") ; Unicode paragraph separator
|
|
|
|
(defun org-babel-execute:elixir (body params)
|
|
(let ((session (cdr (assoc :session params)))
|
|
(tmp (org-babel-temp-file "elixir-")))
|
|
(ob-elixir-ensure-session session params)
|
|
(with-temp-file tmp (insert body))
|
|
(ob-elixir-eval session (format "import_file(\"%s\")" tmp))))
|
|
|
|
(defun ob-elixir-eval (session body)
|
|
(let ((result (ob-elixir-eval-in-repl session body)))
|
|
;; Heavy regex cleanup of IEx output
|
|
(replace-regexp-in-string
|
|
"^\\(import_file([^)]+)\\)+\n" ""
|
|
(replace-regexp-in-string
|
|
"\r" ""
|
|
(replace-regexp-in-string
|
|
"\n\\(\\(iex\\|[.]+\\)\\(([^@]+@[^)]+)[0-9]+\\|([0-9]+)\\)> \\)+" ""
|
|
(replace-regexp-in-string
|
|
"\e\\[[0-9;]*[A-Za-z]" ""
|
|
(replace-regexp-in-string
|
|
"\"\\\\u2029\"" ""
|
|
result)))))))
|
|
```
|
|
|
|
### Strengths
|
|
|
|
| Feature | Description |
|
|
|---------------------|--------------------------------------------------------|
|
|
| **Session support** | Uses IEx sessions for persistent state |
|
|
| **Remote shell** | Supports `--remsh` for connecting to running nodes |
|
|
| **Node naming** | Supports `--sname` and `--name` for distributed Erlang |
|
|
| **Cookie support** | Can specify Erlang cookie for authentication |
|
|
| **Simple design** | Minimal code, easy to understand |
|
|
|
|
### Weaknesses
|
|
|
|
| Issue | Description | Impact |
|
|
|----------------------------|-----------------------------------------------------|----------------------------|
|
|
| **Always uses sessions** | No one-shot execution mode | Slow for simple scripts |
|
|
| **No `:results value`** | Only captures output, not return values | Limited functionality |
|
|
| **No `:var` support** | Cannot pass variables to code blocks | Poor org-babel integration |
|
|
| **Fragile output parsing** | Multiple regex replacements to clean IEx output | Unreliable |
|
|
| **Uses `import_file`** | Relies on IEx-specific command | Tight coupling to IEx |
|
|
| **Global state** | Uses `ob-elixir-process-output` global variable | Not thread-safe |
|
|
| **No Mix support** | Cannot execute in Mix project context | Limited for real projects |
|
|
| **Hardcoded `sit-for`** | Uses fixed delays instead of proper synchronization | Timing issues |
|
|
| **No error handling** | Errors not properly detected or reported | Poor UX |
|
|
| **No tests** | No test suite | Quality concerns |
|
|
| **No `defcustom`** | Settings not customizable via Customize | Poor UX |
|
|
|
|
### Specific Issues in Code
|
|
|
|
1. **Timing-based synchronization**:
|
|
```elisp
|
|
(sit-for 0.5) ; Wait 500ms and hope IEx is ready
|
|
;; This is fragile - slow systems may fail
|
|
```
|
|
|
|
2. **No prompt detection**:
|
|
```elisp
|
|
(defun ob-elixir-wait ()
|
|
(while (not (string-match-p ob-elixir-eoe ob-elixir-process-output))
|
|
(sit-for 0.2)))
|
|
;; Busy-waiting instead of using process filter properly
|
|
```
|
|
|
|
3. **Heavy output cleanup**:
|
|
```elisp
|
|
;; 5 nested regex replacements - fragile and hard to debug
|
|
(replace-regexp-in-string "pattern1" ""
|
|
(replace-regexp-in-string "pattern2" ""
|
|
(replace-regexp-in-string "pattern3" ""
|
|
...)))
|
|
```
|
|
|
|
---
|
|
|
|
## Comparison with Other ob-* Packages
|
|
|
|
### ob-python (Standard Library)
|
|
|
|
**Features we should match:**
|
|
|
|
```elisp
|
|
;; Multiple execution modes
|
|
(defcustom org-babel-python-command "python3"
|
|
"Command to execute Python code.")
|
|
|
|
;; Proper result handling
|
|
(defun org-babel-execute:python (body params)
|
|
(let* ((session (org-babel-python-initiate-session
|
|
(cdr (assq :session params)) params))
|
|
(result-params (cdr (assq :result-params params)))
|
|
(result-type (cdr (assq :result-type params)))
|
|
(full-body (org-babel-expand-body:generic ...))
|
|
(result (org-babel-python-evaluate ...)))
|
|
;; Proper result formatting
|
|
(org-babel-reassemble-table
|
|
(org-babel-result-cond result-params
|
|
result
|
|
(org-babel-python-table-or-string result))
|
|
...)))
|
|
|
|
;; Variable injection
|
|
(defun org-babel-variable-assignments:python (params)
|
|
(mapcar (lambda (pair)
|
|
(format "%s=%s" (car pair)
|
|
(org-babel-python-var-to-python (cdr pair))))
|
|
(org-babel--get-vars params)))
|
|
|
|
;; Both session and non-session evaluation
|
|
(defun org-babel-python-evaluate (session body &optional result-type result-params)
|
|
(if session
|
|
(org-babel-python-evaluate-session ...)
|
|
(org-babel-python-evaluate-external-process ...)))
|
|
```
|
|
|
|
### ob-ruby (Standard Library)
|
|
|
|
**Patterns to adopt:**
|
|
|
|
```elisp
|
|
;; Clean wrapper for value capture
|
|
(defconst org-babel-ruby-wrapper-method
|
|
"
|
|
def main
|
|
%s
|
|
end
|
|
puts main.inspect
|
|
")
|
|
|
|
;; Proper session management with comint
|
|
(defun org-babel-ruby-initiate-session (&optional session params)
|
|
(unless (string= session "none")
|
|
(let ((session-buffer (or (get-buffer ...) ...)))
|
|
(if (org-babel-comint-buffer-livep session-buffer)
|
|
session-buffer
|
|
(org-babel-ruby-initiate-session session)))))
|
|
|
|
;; Type conversion
|
|
(defun org-babel-ruby-var-to-ruby (var)
|
|
(if (listp var)
|
|
(concat "[" (mapconcat #'org-babel-ruby-var-to-ruby var ", ") "]")
|
|
(format "%S" var)))
|
|
```
|
|
|
|
### ob-shell (Standard Library)
|
|
|
|
**Useful patterns:**
|
|
|
|
```elisp
|
|
;; Multiple shell types
|
|
(defcustom org-babel-shell-names
|
|
'("sh" "bash" "zsh" "fish" ...)
|
|
"Shells to register with org-babel.")
|
|
|
|
;; Header arg for shell selection
|
|
(defconst org-babel-header-args:shell
|
|
'((shebang . :any)))
|
|
|
|
;; Proper prep-session
|
|
(defun org-babel-prep-session:shell (session params)
|
|
(let* ((session (org-babel-sh-initiate-session session))
|
|
(var-lines (org-babel-variable-assignments:shell params)))
|
|
(org-babel-comint-in-buffer session
|
|
(mapc (lambda (var)
|
|
(insert var) (comint-send-input nil t)
|
|
(org-babel-comint-wait-for-output session))
|
|
var-lines))
|
|
session))
|
|
```
|
|
|
|
### Feature Comparison Matrix
|
|
|
|
| Feature | ob-python | ob-ruby | ob-shell | ob-elixir (zweifisch) |
|
|
|--------------------|-----------|---------|----------|-----------------------|
|
|
| One-shot execution | Yes | Yes | Yes | No |
|
|
| Session support | Yes | Yes | Yes | Yes |
|
|
| `:results value` | Yes | Yes | Yes | No |
|
|
| `:results output` | Yes | Yes | Yes | Yes (only) |
|
|
| Variable injection | Yes | Yes | Yes | No |
|
|
| Table output | Yes | Yes | Yes | No |
|
|
| Error handling | Yes | Yes | Yes | No |
|
|
| Customization | Yes | Yes | Yes | No |
|
|
| Tests | Yes | Yes | Yes | No |
|
|
| Graphics support | Yes | No | No | No |
|
|
| Async support | Yes | No | No | No |
|
|
|
|
---
|
|
|
|
## Feature Gap Analysis
|
|
|
|
### Critical Gaps (Must Fix)
|
|
|
|
1. **No `:results value` support**
|
|
- Cannot capture return value of expressions
|
|
- Users must use `IO.inspect` workarounds
|
|
- **Fix**: Implement wrapper method pattern
|
|
|
|
2. **No variable injection**
|
|
- Cannot use `:var` header argument
|
|
- Breaks org-babel's data passing paradigm
|
|
- **Fix**: Implement `org-babel-variable-assignments:elixir`
|
|
|
|
3. **No one-shot execution**
|
|
- Every evaluation starts/uses IEx session
|
|
- Slow for simple scripts
|
|
- **Fix**: Add external process execution mode
|
|
|
|
4. **Fragile output parsing**
|
|
- Multiple regex replacements prone to breaking
|
|
- **Fix**: Use markers or JSON encoding
|
|
|
|
### Important Gaps (Should Fix)
|
|
|
|
1. **No Mix project support**
|
|
- Cannot run code in project context
|
|
- No access to project dependencies
|
|
- **Fix**: Add `:mix-project` header argument
|
|
|
|
2. **No proper error handling**
|
|
- Errors appear as raw output
|
|
- No structured error information
|
|
- **Fix**: Detect and signal Elixir errors
|
|
|
|
3. **No customization**
|
|
- Hardcoded paths and settings
|
|
- **Fix**: Add `defcustom` for all configurable values
|
|
|
|
4. **No tests**
|
|
- No way to verify correctness
|
|
- **Fix**: Add comprehensive test suite
|
|
|
|
### Nice-to-Have Gaps (Could Fix)
|
|
|
|
1. **No async execution**
|
|
- Long-running code blocks UI
|
|
- Consider for future versions
|
|
|
|
2. **No graphics support**
|
|
- Elixir has visualization libraries (VegaLite)
|
|
- Could add in future
|
|
|
|
3. **No LSP integration**
|
|
- Could integrate with ElixirLS for completions
|
|
- Future enhancement
|
|
|
|
---
|
|
|
|
## Recommendations
|
|
|
|
### Approach: New Implementation
|
|
|
|
Given the significant gaps and architectural issues in zweifisch/ob-elixir, we recommend **creating a new implementation** rather than forking. Reasons:
|
|
|
|
1. Need to change fundamental architecture (session-only → dual mode)
|
|
2. Core data flow needs redesign
|
|
3. Limited active maintenance upstream
|
|
4. Cleaner API design possible with fresh start
|
|
|
|
### Proposed Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ org-babel-execute:elixir │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌──────────┴──────────┐
|
|
│ │
|
|
▼ ▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ Session Mode │ │ External Mode │
|
|
│ (IEx/comint) │ │ (elixir cmd) │
|
|
└─────────────────┘ └─────────────────┘
|
|
│ │
|
|
▼ ▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ With Mix │ │ With Mix │
|
|
│ (iex -S mix) │ │ (mix run) │
|
|
└─────────────────┘ └─────────────────┘
|
|
```
|
|
|
|
### Implementation Phases
|
|
|
|
#### Phase 1: Core (MVP)
|
|
- [x] One-shot execution with `elixir` command
|
|
- [x] `:results value` and `:results output` support
|
|
- [x] Variable injection with `:var`
|
|
- [x] Basic error detection
|
|
- [x] Customizable commands
|
|
- [x] Test suite foundation
|
|
|
|
#### Phase 2: Sessions
|
|
- [ ] IEx session support via comint
|
|
- [ ] Session persistence
|
|
- [ ] Proper prompt detection
|
|
- [ ] Session cleanup
|
|
|
|
#### Phase 3: Mix Integration
|
|
- [ ] `:mix-project` header argument
|
|
- [ ] Automatic Mix project detection
|
|
- [ ] `iex -S mix` for sessions
|
|
- [ ] `mix run` for one-shot
|
|
|
|
#### Phase 4: Advanced Features
|
|
- [ ] Remote shell (remsh) support
|
|
- [ ] Table input/output
|
|
- [ ] Async execution
|
|
- [ ] Better error messages with line numbers
|
|
|
|
### Header Arguments to Support
|
|
|
|
```elisp
|
|
(defconst org-babel-header-args:elixir
|
|
'(;; Standard org-babel args work automatically
|
|
;; Language-specific:
|
|
(mix-project . :any) ; Path to mix.exs directory
|
|
(mix-env . :any) ; MIX_ENV (dev, test, prod)
|
|
(iex-args . :any) ; Extra IEx arguments
|
|
(timeout . :any) ; Execution timeout
|
|
(node-name . :any) ; --name for distributed
|
|
(node-sname . :any) ; --sname for distributed
|
|
(cookie . :any) ; Erlang cookie
|
|
(remsh . :any)) ; Remote shell node
|
|
"Elixir-specific header arguments.")
|
|
```
|
|
|
|
### Example Usage (Target)
|
|
|
|
```org
|
|
* Basic Evaluation
|
|
|
|
#+BEGIN_SRC elixir
|
|
Enum.map([1, 2, 3], &(&1 * 2))
|
|
#+END_SRC
|
|
|
|
#+RESULTS:
|
|
| 2 | 4 | 6 |
|
|
|
|
* With Variables
|
|
|
|
#+NAME: my-data
|
|
| a | 1 |
|
|
| b | 2 |
|
|
|
|
#+BEGIN_SRC elixir :var data=my-data
|
|
Enum.map(data, fn [k, v] -> {k, v * 10} end)
|
|
#+END_SRC
|
|
|
|
#+RESULTS:
|
|
| a | 10 |
|
|
| b | 20 |
|
|
|
|
* In Mix Project
|
|
|
|
#+BEGIN_SRC elixir :mix-project ~/my_app :mix-env test
|
|
MyApp.some_function()
|
|
#+END_SRC
|
|
|
|
* With Session
|
|
|
|
#+BEGIN_SRC elixir :session my-session
|
|
defmodule Helper do
|
|
def double(x), do: x * 2
|
|
end
|
|
#+END_SRC
|
|
|
|
#+BEGIN_SRC elixir :session my-session
|
|
Helper.double(21)
|
|
#+END_SRC
|
|
|
|
#+RESULTS:
|
|
: 42
|
|
|
|
* Capturing Output
|
|
|
|
#+BEGIN_SRC elixir :results output
|
|
IO.puts("Hello")
|
|
IO.puts("World")
|
|
#+END_SRC
|
|
|
|
#+RESULTS:
|
|
: Hello
|
|
: World
|
|
```
|
|
|
|
---
|
|
|
|
## References
|
|
|
|
- [zweifisch/ob-elixir](https://github.com/zweifisch/ob-elixir)
|
|
- [Org Mode ob-python.el](https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/ob-python.el)
|
|
- [Org Mode ob-ruby.el](https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/ob-ruby.el)
|
|
- [Org Mode ob-shell.el](https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/ob-shell.el)
|
|
- [Org Mode ob-emacs-lisp.el](https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/ob-emacs-lisp.el)
|
|
- [elixir-editors/emacs-elixir](https://github.com/elixir-editors/emacs-elixir)
|
|
- [Worg Babel Languages](https://orgmode.org/worg/org-contrib/babel/languages/)
|