luis/ob-elixir
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
ob-elixir/tasks/08-mix-project-support.md
Luis Eduardo Bueso de Barrio 293d97884c update
2026-01-24 19:27:55 +01:00

735 lines
23 KiB
Markdown
Raw Permalink Blame History

# Task 08: Mix Dependencies Support
**Phase**: 3 - Mix Integration
**Priority**: High
**Estimated Time**: 3-4 hours
**Dependencies**: Task 07 (Session Support) or Phase 1 complete
## Objective
Implement Mix dependencies support so users can specify dependencies directly in the org file using a special `#+BEGIN_DEPS` block. ob-elixir will automatically create a temporary Mix project with those dependencies, cache it for reuse, and use it as the execution context for subsequent Elixir code blocks.
## Prerequisites
- Phase 1 complete (or Phase 2 for session+deps)
- Understanding of Mix build tool
- Network access for fetching dependencies
## Background
### The Problem
When writing Elixir code in org-mode, users often need access to external libraries (e.g., JSON parsing, HTTP clients, custom libraries). Without Mix project support, users are limited to Elixir's standard library.
### The Solution
Instead of requiring users to point to an existing Mix project, ob-elixir will:
1. Parse dependency specifications from a special `#+BEGIN_DEPS` block
2. Create a temporary Mix project with those dependencies
3. Cache the project based on a hash of the dependencies (for fast re-evaluation)
4. Execute code blocks within that project context
5. Clean up temporary projects when Emacs exits
### Example Usage
```org
* Using External Dependencies
#+BEGIN_DEPS elixir
[
{:jason, "~> 1.4"},
{:prova, git: "git@gitlab.com:babel-upm/makina/prova.git"}
]
#+END_DEPS
#+BEGIN_SRC elixir
# Both jason and prova are available
Jason.encode!(%{hello: "world"})
#+END_SRC
#+BEGIN_SRC elixir :session deps-session
# Session also has access to deps
Prova.some_function()
#+END_SRC
* Different Dependencies Section
#+BEGIN_DEPS elixir
[{:httpoison, "~> 2.0"}]
#+END_DEPS
#+BEGIN_SRC elixir
# Now using httpoison instead of the previous deps
HTTPoison.get!("https://example.com")
#+END_SRC
```
## Design Decisions
| Aspect | Decision | Rationale |
|--------|----------|-----------|
| Block format | `#+BEGIN_DEPS elixir ... #+END_DEPS` | Special block, not confused with executable code |
| Deps syntax | Elixir list: `[{:pkg, "~> 1.0"}]` | Natural for Elixir users, supports all dep formats |
| Scope | All subsequent blocks until next deps block | Simple mental model |
| Caching | Hash-based | Fast re-evaluation, no redundant compilation |
| Multiple deps blocks | Later overrides | Allows different sections with different deps |
| Cleanup | Auto on Emacs exit | No orphaned temp projects |
| Elixir version | System default | Keep it simple, no version management |
## Steps
### Step 1: Add configuration and infrastructure
```elisp
;;; Mix Dependencies Configuration
(defcustom ob-elixir-mix-command "mix"
"Command to run Mix."
:type 'string
:group 'ob-elixir
:safe #'stringp)
(defcustom ob-elixir-deps-cache-dir
(expand-file-name "ob-elixir" (or (getenv "XDG_CACHE_HOME") "~/.cache"))
"Directory for caching temporary Mix projects.
Each unique set of dependencies gets its own subdirectory,
named by the hash of the dependencies."
:type 'directory
:group 'ob-elixir)
(defvar ob-elixir--deps-projects (make-hash-table :test 'equal)
"Hash table mapping deps-hash to project directory path.")
(defvar ob-elixir--cleanup-registered nil
"Whether the cleanup hook has been registered.")
```
### Step 2: Implement deps block parsing
```elisp
;;; Deps Block Parsing
(defconst ob-elixir--deps-block-regexp
"^[ \t]*#\\+BEGIN_DEPS[ \t]+elixir[ \t]*\n\\(\\(?:.*\n\\)*?\\)[ \t]*#\\+END_DEPS"
"Regexp matching a deps block.
Group 1 captures the deps content.")
(defun ob-elixir--find-deps-for-position (pos)
"Find the most recent deps block before POS.
Returns the deps content as a string, or nil if no deps block found."
(save-excursion
(goto-char pos)
(let ((found nil))
;; Search backward for deps blocks
(while (and (not found)
(re-search-backward ob-elixir--deps-block-regexp nil t))
(when (< (match-end 0) pos)
(setq found (match-string-no-properties 1))))
found)))
(defun ob-elixir--normalize-deps (deps-string)
"Normalize DEPS-STRING for consistent hashing.
Removes comments, extra whitespace, and normalizes formatting."
(let ((normalized deps-string))
;; Remove Elixir comments
(setq normalized (replace-regexp-in-string "#.*$" "" normalized))
;; Normalize whitespace
(setq normalized (replace-regexp-in-string "[ \t\n]+" " " normalized))
;; Trim
(string-trim normalized)))
(defun ob-elixir--hash-deps (deps-string)
"Compute SHA256 hash of DEPS-STRING for caching."
(secure-hash 'sha256 (ob-elixir--normalize-deps deps-string)))
```
### Step 3: Implement temp Mix project creation
```elisp
;;; Temporary Mix Project Management
(defconst ob-elixir--mix-exs-template
"defmodule ObElixirTemp.MixProject do
use Mix.Project
def project do
[
app: :ob_elixir_temp,
version: \"0.1.0\",
elixir: \"~> 1.14\",
start_permanent: false,
deps: deps()
]
end
def application do
[extra_applications: [:logger]]
end
defp deps do
%s
end
end
"
"Template for temporary Mix project mix.exs file.
%s is replaced with the deps list.")
(defun ob-elixir--ensure-cache-dir ()
"Ensure the cache directory exists."
(unless (file-directory-p ob-elixir-deps-cache-dir)
(make-directory ob-elixir-deps-cache-dir t)))
(defun ob-elixir--get-deps-project (deps-string)
"Get or create a Mix project for DEPS-STRING.
Returns the project directory path, creating and initializing
the project if necessary."
(let* ((deps-hash (ob-elixir--hash-deps deps-string))
(cached (gethash deps-hash ob-elixir--deps-projects)))
(if (and cached (file-directory-p cached))
cached
;; Create new project
(ob-elixir--create-deps-project deps-hash deps-string))))
(defun ob-elixir--create-deps-project (deps-hash deps-string)
"Create a new temporary Mix project for DEPS-STRING.
DEPS-HASH is used as the directory name.
Returns the project directory path."
(ob-elixir--ensure-cache-dir)
(ob-elixir--register-cleanup)
(let* ((project-dir (expand-file-name deps-hash ob-elixir-deps-cache-dir))
(mix-exs-path (expand-file-name "mix.exs" project-dir))
(lib-dir (expand-file-name "lib" project-dir)))
;; Create project structure
(make-directory project-dir t)
(make-directory lib-dir t)
;; Write mix.exs
(with-temp-file mix-exs-path
(insert (format ob-elixir--mix-exs-template deps-string)))
;; Write a placeholder module
(with-temp-file (expand-file-name "ob_elixir_temp.ex" lib-dir)
(insert "defmodule ObElixirTemp do\nend\n"))
;; Fetch dependencies
(let ((default-directory project-dir))
(message "ob-elixir: Fetching dependencies...")
(let ((output (shell-command-to-string
(format "%s deps.get 2>&1" ob-elixir-mix-command))))
(when (string-match-p "\\*\\*.*error\\|Could not" output)
(error "ob-elixir: Failed to fetch dependencies:\n%s" output)))
;; Compile
(message "ob-elixir: Compiling dependencies...")
(let ((output (shell-command-to-string
(format "%s compile 2>&1" ob-elixir-mix-command))))
(when (string-match-p "\\*\\*.*[Ee]rror" output)
(error "ob-elixir: Failed to compile dependencies:\n%s" output))))
;; Cache and return
(puthash deps-hash project-dir ob-elixir--deps-projects)
(message "ob-elixir: Dependencies ready")
project-dir))
```
### Step 4: Implement execution with deps context
```elisp
;;; Execution with Dependencies
(defun ob-elixir--execute-with-deps (body result-type deps-string)
"Execute BODY with dependencies from DEPS-STRING.
RESULT-TYPE is 'value or 'output."
(let* ((project-dir (ob-elixir--get-deps-project deps-string))
(default-directory project-dir)
(tmp-file (org-babel-temp-file "ob-elixir-" ".exs"))
(code (if (eq result-type 'value)
(ob-elixir--wrap-for-value body)
body)))
;; Write code to temp file
(with-temp-file tmp-file
(insert code))
;; Execute with mix run
(let ((command (format "%s run --no-compile %s 2>&1"
ob-elixir-mix-command
(shell-quote-argument tmp-file))))
(ob-elixir--process-result
(shell-command-to-string command)))))
(defun ob-elixir--start-session-with-deps (buffer-name session-name deps-string)
"Start a new IEx session with dependencies in BUFFER-NAME.
SESSION-NAME is used for the process name.
DEPS-STRING contains the dependencies specification."
(let* ((project-dir (ob-elixir--get-deps-project deps-string))
(buffer (get-buffer-create buffer-name))
(default-directory project-dir)
(process-environment (cons "TERM=dumb" process-environment)))
(with-current-buffer buffer
;; Start IEx with Mix project
(make-comint-in-buffer
(format "ob-elixir-%s" session-name)
buffer
ob-elixir-iex-command
nil
"-S" "mix")
;; Wait for initial prompt
(ob-elixir--wait-for-prompt buffer 30) ; Longer timeout for deps loading
;; Configure IEx for programmatic use
(ob-elixir--configure-session buffer)
buffer)))
```
### Step 5: Update main execute function
Modify `org-babel-execute:elixir` to check for deps context:
```elisp
(defun org-babel-execute:elixir (body params)
"Execute a block of Elixir code with org-babel.
BODY is the Elixir code to execute.
PARAMS is an alist of header arguments."
(let* ((session (cdr (assq :session params)))
(result-type (cdr (assq :result-type params)))
(result-params (cdr (assq :result-params params)))
;; Find deps for this block's position
(deps-string (ob-elixir--find-deps-for-position (point)))
;; Expand body with variable assignments
(full-body (org-babel-expand-body:generic
body params
(org-babel-variable-assignments:elixir params)))
(result (cond
;; Session mode with deps
((and session (not (string= session "none")) deps-string)
(ob-elixir--evaluate-in-session-with-deps
session full-body result-type deps-string))
;; Session mode without deps
((and session (not (string= session "none")))
(ob-elixir--evaluate-in-session session full-body result-type))
;; Non-session with deps
(deps-string
(ob-elixir--execute-with-deps full-body result-type deps-string))
;; Plain execution
(t
(ob-elixir--execute full-body result-type)))))
(org-babel-reassemble-table
(org-babel-result-cond result-params
result
(ob-elixir--table-or-string result))
(org-babel-pick-name (cdr (assq :colname-names params))
(cdr (assq :colnames params)))
(org-babel-pick-name (cdr (assq :rowname-names params))
(cdr (assq :rownames params))))))
```
### Step 6: Implement session support with deps
```elisp
(defvar ob-elixir--session-deps (make-hash-table :test 'equal)
"Hash table mapping session names to their deps-hash.
Used to ensure session deps consistency.")
(defun ob-elixir--get-or-create-session-with-deps (name deps-string params)
"Get or create an IEx session NAME with DEPS-STRING."
(let* ((deps-hash (ob-elixir--hash-deps deps-string))
(buffer-name (format "*ob-elixir:%s*" name))
(existing (get-buffer buffer-name))
(existing-deps (gethash name ob-elixir--session-deps)))
;; Check if existing session has different deps
(when (and existing existing-deps (not (string= existing-deps deps-hash)))
(message "ob-elixir: Session %s has different deps, recreating..." name)
(ob-elixir-kill-session name)
(setq existing nil))
(if (and existing (org-babel-comint-buffer-livep existing))
existing
;; Create new session
(let ((buffer (ob-elixir--start-session-with-deps buffer-name name deps-string)))
(puthash name deps-hash ob-elixir--session-deps)
buffer))))
(defun ob-elixir--evaluate-in-session-with-deps (session body result-type deps-string)
"Evaluate BODY in SESSION with DEPS-STRING context."
(let* ((buffer (ob-elixir--get-or-create-session-with-deps session deps-string nil))
(code (if (eq result-type 'value)
(ob-elixir--session-wrap-for-value body)
body))
(eoe-indicator ob-elixir--eoe-marker)
(full-body (concat code "\nIO.puts(\"" eoe-indicator "\")"))
output)
(unless buffer
(error "Failed to create Elixir session with deps: %s" session))
(setq output
(org-babel-comint-with-output
(buffer eoe-indicator t full-body)
(ob-elixir--send-command buffer full-body)))
(ob-elixir--clean-session-output output result-type)))
```
### Step 7: Implement cleanup
```elisp
;;; Cleanup
(defun ob-elixir--register-cleanup ()
"Register cleanup hook if not already registered."
(unless ob-elixir--cleanup-registered
(add-hook 'kill-emacs-hook #'ob-elixir-cleanup-deps-projects)
(setq ob-elixir--cleanup-registered t)))
(defun ob-elixir-cleanup-deps-projects ()
"Delete all temporary Mix projects created by ob-elixir.
Called automatically on Emacs exit."
(interactive)
(maphash
(lambda (_hash project-dir)
(when (and project-dir (file-directory-p project-dir))
(condition-case err
(delete-directory project-dir t)
(error
(message "ob-elixir: Failed to delete %s: %s" project-dir err)))))
ob-elixir--deps-projects)
(clrhash ob-elixir--deps-projects)
(message "ob-elixir: Cleaned up temporary Mix projects"))
(defun ob-elixir-list-deps-projects ()
"List all cached dependency projects."
(interactive)
(if (= (hash-table-count ob-elixir--deps-projects) 0)
(message "No cached dependency projects")
(with-output-to-temp-buffer "*ob-elixir deps projects*"
(princ "Cached ob-elixir dependency projects:\n\n")
(maphash
(lambda (hash dir)
(princ (format " %s\n -> %s\n" (substring hash 0 12) dir)))
ob-elixir--deps-projects))))
```
### Step 8: Add tests
Create `test/test-ob-elixir-deps.el`:
```elisp
;;; test-ob-elixir-deps.el --- Deps block tests -*- lexical-binding: t; -*-
(require 'ert)
(require 'ob-elixir)
;;; Parsing Tests
(ert-deftest ob-elixir-test-deps-block-parsing ()
"Test finding deps blocks in buffer."
(with-temp-buffer
(insert "#+BEGIN_DEPS elixir\n[{:jason, \"~> 1.4\"}]\n#+END_DEPS\n\n")
(insert "#+BEGIN_SRC elixir\nJason.encode!(%{})\n#+END_SRC\n")
(goto-char (point-max))
(let ((deps (ob-elixir--find-deps-for-position (point))))
(should deps)
(should (string-match-p ":jason" deps)))))
(ert-deftest ob-elixir-test-deps-block-override ()
"Test that later deps blocks override earlier ones."
(with-temp-buffer
(insert "#+BEGIN_DEPS elixir\n[{:jason, \"~> 1.4\"}]\n#+END_DEPS\n\n")
(insert "#+BEGIN_SRC elixir\ncode1\n#+END_SRC\n\n")
(insert "#+BEGIN_DEPS elixir\n[{:httpoison, \"~> 2.0\"}]\n#+END_DEPS\n\n")
(let ((pos (point)))
(insert "#+BEGIN_SRC elixir\ncode2\n#+END_SRC\n")
(let ((deps (ob-elixir--find-deps-for-position (+ pos 10))))
(should deps)
(should (string-match-p ":httpoison" deps))
(should-not (string-match-p ":jason" deps))))))
(ert-deftest ob-elixir-test-no-deps-block ()
"Test behavior when no deps block exists."
(with-temp-buffer
(insert "#+BEGIN_SRC elixir\n1 + 1\n#+END_SRC\n")
(should (null (ob-elixir--find-deps-for-position (point))))))
(ert-deftest ob-elixir-test-deps-hash-consistency ()
"Test that same deps produce same hash."
(let ((deps1 "[{:jason, \"~> 1.4\"}]")
(deps2 "[{:jason, \"~> 1.4\"}]") ; extra space
(deps3 "[{:httpoison, \"~> 2.0\"}]"))
(should (string= (ob-elixir--hash-deps deps1)
(ob-elixir--hash-deps deps2)))
(should-not (string= (ob-elixir--hash-deps deps1)
(ob-elixir--hash-deps deps3)))))
(ert-deftest ob-elixir-test-normalize-deps ()
"Test deps normalization."
(should (string= (ob-elixir--normalize-deps "[ {:a, \"1\"} ]")
(ob-elixir--normalize-deps "[{:a,\"1\"}]")))
;; Comments are stripped
(should (string= (ob-elixir--normalize-deps "[{:a, \"1\"}] # comment")
(ob-elixir--normalize-deps "[{:a, \"1\"}]"))))
;;; Integration Tests (require network and mix)
(ert-deftest ob-elixir-test-deps-project-creation ()
"Test creating a temporary Mix project with deps."
(skip-unless (and (executable-find ob-elixir-mix-command)
(executable-find ob-elixir-command)))
(let ((ob-elixir-deps-cache-dir (make-temp-file "ob-elixir-test-" t))
(deps "[{:jason, \"~> 1.4\"}]"))
(unwind-protect
(let ((project-dir (ob-elixir--get-deps-project deps)))
(should (file-directory-p project-dir))
(should (file-exists-p (expand-file-name "mix.exs" project-dir)))
(should (file-directory-p (expand-file-name "deps/jason" project-dir))))
;; Cleanup
(ob-elixir-cleanup-deps-projects)
(delete-directory ob-elixir-deps-cache-dir t))))
(ert-deftest ob-elixir-test-deps-execution ()
"Test executing code with deps."
(skip-unless (and (executable-find ob-elixir-mix-command)
(executable-find ob-elixir-command)))
(let ((ob-elixir-deps-cache-dir (make-temp-file "ob-elixir-test-" t))
(deps "[{:jason, \"~> 1.4\"}]"))
(unwind-protect
(let ((result (ob-elixir--execute-with-deps
"Jason.encode!(%{a: 1})"
'value
deps)))
(should (string-match-p "\"a\"" result)))
;; Cleanup
(ob-elixir-cleanup-deps-projects)
(delete-directory ob-elixir-deps-cache-dir t))))
(ert-deftest ob-elixir-test-deps-caching ()
"Test that deps projects are cached and reused."
(skip-unless (executable-find ob-elixir-mix-command))
(let ((ob-elixir-deps-cache-dir (make-temp-file "ob-elixir-test-" t))
(deps "[{:jason, \"~> 1.4\"}]"))
(unwind-protect
(let ((project1 (ob-elixir--get-deps-project deps))
(project2 (ob-elixir--get-deps-project deps)))
;; Should return same directory
(should (string= project1 project2))
;; Should only have one entry in hash table
(should (= 1 (hash-table-count ob-elixir--deps-projects))))
;; Cleanup
(ob-elixir-cleanup-deps-projects)
(delete-directory ob-elixir-deps-cache-dir t))))
(provide 'test-ob-elixir-deps)
```
### Step 9: Test in org buffer
Create test cases in `test.org`:
```org
* Deps Block Tests
** Basic deps usage
#+BEGIN_DEPS elixir
[{:jason, "~> 1.4"}]
#+END_DEPS
#+BEGIN_SRC elixir
Jason.encode!(%{hello: "world", number: 42})
#+END_SRC
** Multiple dependencies
#+BEGIN_DEPS elixir
[
{:jason, "~> 1.4"},
{:decimal, "~> 2.0"}
]
#+END_DEPS
#+BEGIN_SRC elixir
num = Decimal.new("3.14159")
Jason.encode!(%{pi: Decimal.to_string(num)})
#+END_SRC
** Git dependencies
#+BEGIN_DEPS elixir
[
{:prova, git: "git@gitlab.com:babel-upm/makina/prova.git"}
]
#+END_DEPS
#+BEGIN_SRC elixir
# Use the prova library
Prova.hello()
#+END_SRC
** Session with deps
#+BEGIN_DEPS elixir
[{:jason, "~> 1.4"}]
#+END_DEPS
#+BEGIN_SRC elixir :session json-session
data = %{users: [%{name: "Alice"}, %{name: "Bob"}]}
#+END_SRC
#+BEGIN_SRC elixir :session json-session
Jason.encode!(data, pretty: true)
#+END_SRC
** Deps override
#+BEGIN_DEPS elixir
[{:httpoison, "~> 2.0"}]
#+END_DEPS
#+BEGIN_SRC elixir
# Jason no longer available here, httpoison is
{:ok, resp} = HTTPoison.get("https://httpbin.org/get")
resp.status_code
#+END_SRC
** No deps (plain execution)
#+BEGIN_SRC elixir
# This block has no deps context (before any deps block or in different file)
Enum.map(1..5, &(&1 * 2))
#+END_SRC
```
## Acceptance Criteria
- [ ] `#+BEGIN_DEPS elixir` block is recognized and parsed
- [ ] Deps content is extracted correctly (Elixir list syntax)
- [ ] Temp Mix project is created with correct `mix.exs`
- [ ] `mix deps.get` runs automatically
- [ ] `mix compile` runs automatically
- [ ] Subsequent elixir blocks can use the dependencies
- [ ] Projects are cached by deps hash (identical deps = same project)
- [ ] Cache reuse is fast (no re-compilation)
- [ ] Sessions work with deps context (`iex -S mix`)
- [ ] Session maintains deps consistency (recreates if deps change)
- [ ] Later deps blocks override earlier ones for subsequent code blocks
- [ ] Auto-cleanup on Emacs exit works
- [ ] `ob-elixir-cleanup-deps-projects` command available
- [ ] `ob-elixir-list-deps-projects` command available
- [ ] Proper error handling for:
- Invalid dependency specs
- Network failures during `deps.get`
- Compilation errors
- [ ] All tests pass
## Header Arguments Reference
This feature does not add new header arguments. Instead, it uses special blocks:
| Block | Format | Description |
|-------|--------|-------------|
| `#+BEGIN_DEPS elixir` | Elixir list | Dependencies for subsequent code blocks |
Supported dependency formats (same as Mix):
```elixir
# Hex package
{:package_name, "~> 1.0"}
# Git repository
{:package_name, git: "https://github.com/user/repo.git"}
{:package_name, git: "git@github.com:user/repo.git", tag: "v1.0"}
{:package_name, git: "...", branch: "main"}
# Path (for local development)
{:package_name, path: "../local_package"}
# With options
{:package_name, "~> 1.0", only: :dev}
{:package_name, "~> 1.0", runtime: false}
```
## Troubleshooting
### Dependencies not found
Ensure the deps block is before the code block:
```org
#+BEGIN_DEPS elixir <-- Must come first
[{:jason, "~> 1.4"}]
#+END_DEPS
#+BEGIN_SRC elixir <-- Code block after deps
Jason.encode!(%{})
#+END_SRC
```
### Network errors during deps.get
Check network connectivity. The error message will include Mix output.
### Compilation errors
Check that dependency versions are compatible. Look at the error output for details.
### Slow first execution
The first execution with new deps will be slow (fetching + compiling). Subsequent executions with the same deps use the cached project.
### Cleanup
To manually clean up cached projects:
```
M-x ob-elixir-cleanup-deps-projects
```
To see what's cached:
```
M-x ob-elixir-list-deps-projects
```
## Files Modified
- `ob-elixir.el` - Add deps block support
- `test/test-ob-elixir-deps.el` - Add deps tests
## Implementation Notes
### Why not use header arguments?
The deps list can be quite long and complex (multiple deps, git URLs, options). A dedicated block is more readable and maintainable than cramming it into a header argument.
### Why auto-cleanup?
Temporary projects can consume significant disk space (compiled deps). Auto-cleanup on exit prevents orphaned projects while still allowing cache reuse during a session.
### Why hash-based caching?
Computing a hash of the normalized deps string is fast and ensures that:
1. Identical deps always use the same cached project
2. Any change in deps (version, options, etc.) creates a new project
3. No false cache hits from similar-looking deps
## References
- [Mix Documentation](https://hexdocs.pm/mix/Mix.html)
- [Mix.Project deps](https://hexdocs.pm/mix/Mix.Tasks.Deps.html)
Reference in New Issue View Git Blame Copy Permalink