/mnt/DATA/WORK/agent-gui ici je build une GUI pour mes agent IA, pour qu'ils aient un avatar et que je puisse les modeler comme je veux. est ce qu'on pourrait mettre ça dessus? comme ça les avatars pourraient parler ? https://github.com/aigc3d/LAM


Intégrer LAM pour avatars parlants dans GUI agent IA


Je vais explorer votre projet et regarder ce que LAM propose.


Bash

find /mnt/DATA/WORK/agent-gui -type f | head -60 && echo "---" && ls /mnt/DATA/WORK/agent-gui

Explore agent-gui project structure


Agent

Research LAM GitHub repo capabilities

Prompt:
Go to https://github.com/aigc3d/LAM and read the README carefully. I need to know:
1. What exactly LAM does (Large Animation Model — is it for 3D avatar animation from audio/text?)
2. What inputs it takes (audio? text? video?)
3. What outputs it produces (3D mesh? video? real-time?)
4. What are the system requirements (GPU, CUDA version, etc.)
5. Is there a Python API or a web demo?
6. Can it be integrated into a web/desktop GUI app?
7. Any licensing info

Report concisely, focusing on integration feasibility.


/mnt/DATA/WORK/agent-gui/PLAN.md
/mnt/DATA/WORK/agent-gui/README.md
/mnt/DATA/WORK/agent-gui/agent_gui/adapters.py
/mnt/DATA/WORK/agent-gui/agent_gui/app.py
/mnt/DATA/WORK/agent-gui/agent_gui/avatar.py
/mnt/DATA/WORK/agent-gui/agent_gui/config.py
/mnt/DATA/WORK/agent-gui/agent_gui/sessions.py
/mnt/DATA/WORK/agent-gui/agent_gui/__init__.py
/mnt/DATA/WORK/agent-gui/configs/agents.json
/mnt/DATA/WORK/agent-gui/static/app.js
/mnt/DATA/WORK/agent-gui/static/index.html
/mnt/DATA/WORK/agent-gui/static/styles.css
/mnt/DATA/WORK/agent-gui/tests/test_adapters.py
/mnt/DATA/WORK/agent-gui/tests/test_avatar.py
/mnt/DATA/WORK/agent-gui/tests/test_config.py
/mnt/DATA/WORK/agent-gui/tests/test_sessions.py
/mnt/DATA/WORK/agent-gui/tests/__pycache__/test_adapters.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/tests/__pycache__/test_avatar.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/tests/__pycache__/test_config.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/tests/__pycache__/test_sessions.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/agent_gui/__pycache__/adapters.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/agent_gui/__pycache__/app.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/agent_gui/__pycache__/avatar.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/agent_gui/__pycache__/config.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/agent_gui/__pycache__/sessions.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/agent_gui/__pycache__/__init__.cpython-314.pyc
/mnt/DATA/WORK/agent-gui/data/sessions/6dd3753b2acf.jsonl
/mnt/DATA/WORK/agent-gui/data/sessions/7772cd958587.jsonl
/mnt/DATA/WORK/agent-gui/data/sessions/8ed5cc84d704.jsonl
/mnt/DATA/WORK/agent-gui/data/sessions/c2ba25240ba9.jsonl
/mnt/DATA/WORK/agent-gui/data/sessions/claude-test-1.jsonl
/mnt/DATA/WORK/agent-gui/data/sessions/e5218deff440.jsonl
---
PLAN.md
README.md
agent_gui
configs
data
static
tests


Async agent launched successfully.
agentId: ac89a0f264ece3ae0 (internal ID - do not mention to user. Use SendMessage with to: 'ac89a0f264ece3ae0' to continue this agent.)
The agent is working in the background. You will be notified automatically when it completes.
Do not duplicate this agent's work — avoid working with the same files or topics it is using. Work on non-overlapping tasks, or briefly tell the user what you launched and end your response.
output_file: /tmp/claude-1000/-home-wildlama/7de9b929-9050-430b-9b36-f74849c1c0a7/tasks/ac89a0f264ece3ae0.output
Do NOT Read or tail this file via the shell tool — it is the full subagent JSONL transcript and reading it will overflow your context. If the user asks for progress, say the agent is still running; you'll get a completion notification.

Research LAM GitHub repo capabilities

Prompt:
Go to https://github.com/aigc3d/LAM and read the README carefully. I need to know:
1. What exactly LAM does (Large Animation Model — is it for 3D avatar animation from audio/text?)
2. What inputs it takes (audio? text? video?)
3. What outputs it produces (3D mesh? video? real-time?)
4. What are the system requirements (GPU, CUDA version, etc.)
5. Is there a Python API or a web demo?
6. Can it be integrated into a web/desktop GUI app?
7. Any licensing info

Report concisely, focusing on integration feasibility.


claude claude-code-guide Explore general-purpose Plan statusline-setup

- claude: Catch-all for any task that doesn't fit a more specific agent. FleetView's default when no agent name is typed. (Tools: *)

- claude-code-guide: Use this agent when the user asks questions ("Can Claude...", "Does Claude...", "How do I...") about: (1) Claude Code (the CLI tool) - features, hooks, slash commands, MCP servers, settings, IDE integrations, keyboard shortcuts; (2) Claude Agent SDK - building custom agents; (3) Claude API (formerly Anthropic API) - API usage, tool use, Anthropic SDK usage. **IMPORTANT:** Before spawning a new agent, check if there is already a running or recently completed claude-code-guide agent that you can continue via SendMessage. (Tools: Bash, Read, WebFetch, WebSearch)

- Explore: Fast read-only search agent for locating code. Use it to find files by pattern (eg. "src/components/**/*.tsx"), grep for symbols or keywords (eg. "API endpoints"), or answer "where is X defined / which files reference Y." Do NOT use it for code review, design-doc auditing, cross-file consistency checks, or open-ended analysis — it reads excerpts rather than whole files and will miss content past its read window. When calling, specify search breadth: "quick" for a single targeted lookup, "medium" for moderate exploration, or "very thorough" to search across multiple locations and naming conventions. (Tools: All tools except Agent, ExitPlanMode, Edit, Write, NotebookEdit)

- general-purpose: General-purpose agent for researching complex questions, searching for code, and executing multi-step tasks. When you are searching for a keyword or file and are not confident that you will find the right match in the first few tries use this agent to perform the search for you. (Tools: *)

- Plan: Software architect agent for designing implementation plans. Use this when you need to plan the implementation strategy for a task. Returns step-by-step plans, identifies critical files, and considers architectural trade-offs. (Tools: All tools except Agent, ExitPlanMode, Edit, Write, NotebookEdit)

- statusline-setup: Use this agent to configure the user's Claude Code status line setting. (Tools: Read, Edit)


- deep-research: Deep research harness — fan-out web searches, fetch sources, adversarially verify claims, synthesize a cited report. - When the user wants a deep, multi-source, fact-checked research report on any topic. BEFORE invoking, check if the question is specific enough to research directly — if underspecified (e.g., "what car to buy" without budget/use-case/region), ask 2-3 clarifying questions to narrow scope. Then pass the refined question as args, weaving the answers in.
- update-config: Use this skill to configure the Claude Code harness via settings.json. Automated behaviors ("from now on when X", "each time X", "whenever X", "before/after X") require hooks configured in settings.json - the harness executes these, not Claude, so memory/preferences cannot fulfill them. Also use for: permissions ("allow X", "add permission", "move permission to"), env vars ("set X=Y"), hook troubleshooting, or any changes to settings.json/settings.local.json files. Examples: "allow npm commands", "add bq permission to global settings", "move permission to user settings", "set DEBUG=true", "when claude stops show X". For simple settings like theme/model, suggest the /config command.
- keybindings-help: Use when the user wants to customize keyboard shortcuts, rebind keys, add chord bindings, or modify ~/.claude/keybindings.json. Examples: "rebind ctrl+s", "add a chord shortcut", "change the submit key", "customize keybindings".
- verify: Verify that a code change actually does what it's supposed to by running the app and observing behavior. Use when asked to verify a PR, confirm a fix works, test a change manually, check that a feature works, or validate local changes before pushing.
- code-review: Review the current diff for correctness bugs and reuse/simplification/efficiency cleanups at the given effort level (low/medium: fewer, high-confidence findings; high→max: broader coverage, may include uncertain findings). Pass --comment to post findings as inline PR comments, or --fix to apply the findings to the working tree after the review.
- simplify: Review the changed code for reuse, simplification, efficiency, and altitude cleanups, then apply the fixes. Quality only — it does not hunt for bugs; use /code-review for that.
- fewer-permission-prompts: Scan your transcripts for common read-only Bash and MCP tool calls, then add a prioritized allowlist to project .claude/settings.json to reduce permission prompts.
- loop: Run a prompt or slash command on a recurring interval (e.g. /loop 5m /foo). Omit the interval to let the model self-pace. - When the user wants to set up a recurring task, poll for status, or run something repeatedly on an interval (e.g. "check the deploy every 5 minutes", "keep running /babysit-prs"). Do NOT invoke for one-off tasks.
- schedule: Create, update, list, or run scheduled cloud agents (routines) that execute on a cron schedule. - When the user wants to schedule a recurring cloud agent, set up automated tasks, create a cron job for Claude Code, or manage their scheduled agents/routines. Also use when the user wants a one-time scheduled run ("run this once at 3pm", "remind me to check X tomorrow").
- claude-api: Reference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration.
TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens).
SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file).
- run: Launch and drive this project's app to see a change working. Use when asked to run, start, or screenshot the app, or to confirm a change works in the real app (not just tests). First looks for a project skill that already covers launching the app; otherwise falls back to built-in patterns per project type (CLI, server, TUI, Electron, browser-driven, library).
- init: Initialize a new CLAUDE.md file with codebase documentation
- review: Review a pull request
- security-review: Complete a security review of the pending changes on the current branch

deep-research update-config keybindings-help verify code-review simplify fewer-permission-prompts loop schedule claude-api run init review security-review


Je lis les fichiers clés du projet pendant que l'agent recherche LAM.


1	# Agent GUI Local
2
3	Web app locale qui sert de surcouche graphique universelle pour agents LLM CLI.
4
5	V1 incluse:
6
7	- Backend Python sans dépendance externe.
8	- Frontend HTML/CSS/JS vanilla.
9	- Chat local.
10	- Sessions JSONL.
11	- Adaptateur Echo pour tester sans modèle.
12	- Adaptateur CLI universel pour brancher Hermes, Claude Code, OpenClaw, etc.
13	- Bureau pixel art animé avec petits personnages façon arcade.
14	- Avatar de bureau en mode sprite, avec états: idle, thinking, tool, speaking, error.
15
16	## Lancer
17
18	```bash
19	cd /home/wildlama/agent-gui
20	python3 -m agent_gui.app --host 127.0.0.1 --port 8765
21	```
22
23	Puis ouvrir:
24
25	```text
26	http://127.0.0.1:8765
27	```
28
29	## Tester
30
31	```bash
32	cd /home/wildlama/agent-gui
33	python3 -m unittest discover -s tests -v
34	```
35
36	Smoke-test API:
37
38	```bash
39	curl http://127.0.0.1:8765/health
40	curl -X POST http://127.0.0.1:8765/api/chat \
41	  -H 'Content-Type: application/json' \
42	  -d '{"agent":"echo","message":"hello"}'
43	```
44
45	## Ajouter un agent
46
47	Éditer:
48
49	```text
50	/home/wildlama/agent-gui/configs/agents.json
51	```
52
53	Exemple:
54
55	```json
56	{
57	  "id": "mon-agent",
58	  "name": "Mon Agent",
59	  "type": "cli",
60	  "description": "Agent CLI custom",
61	  "command": ["mon-agent", "run", "{prompt}"],
62	  "timeout_seconds": 300,
63	  "enabled": true
64	}
65	```
66
67	Règles:
68
69	- `{prompt}` est remplacé par le message utilisateur.
70	- `{session_id}` peut être utilisé si l'agent accepte un identifiant de session.
71	- La commande est une liste d'arguments, pas une string shell.
72	- Le backend n'utilise pas `shell=True`, pour éviter les injections shell.
73
74	## Agents préconfigurés
75
76	- `echo`: intégré, marche toujours.
77	- `hermes`: `hermes chat -q {prompt}`.
78	- `claude-code`: `claude -p --output-format text --permission-mode default {prompt}`.
79	- `openclaw`: exemple désactivé, à ajuster selon ton installation.
80
81	## Limites V1
82
83	- Pas encore de streaming token par token.
84	- Pas encore d'approbations interactives dans l'UI.
85	- Les agents CLI sont appelés en one-shot.
86	- Les sorties tool calls ne sont pas encore parsées profondément, elles sont affichées comme messages ou status.
87
88	## Suite logique
89
90	- Ajouter SSE ou WebSocket pour streaming temps réel.
91	- Ajouter des sprites 2D par état à la place de l'ASCII.
92	- Ajouter un protocole d'événements JSON optionnel pour agents compatibles.
93	- Ajouter un adaptateur Hermes profond via API gateway ou ACP.
94	- Ajouter un panneau fichiers / workspace.
95	- Ajouter approvals UI pour commandes dangereuses.
96
97	Voir aussi:
98
99	```text
100	/home/wildlama/agent-gui/PLAN.md
101	```
102

# Agent GUI Local

Web app locale qui sert de surcouche graphique universelle pour agents LLM CLI.

V1 incluse:

- Backend Python sans dépendance externe.
- Frontend HTML/CSS/JS vanilla.
- Chat local.
- Sessions JSONL.
- Adaptateur Echo pour tester sans modèle.
- Adaptateur CLI universel pour brancher Hermes, Claude Code, OpenClaw, etc.
- Bureau pixel art animé avec petits personnages façon arcade.
- Avatar de bureau en mode sprite, avec états: idle, thinking, tool, speaking, error.

## Lancer

```bash
cd /home/wildlama/agent-gui
python3 -m agent_gui.app --host 127.0.0.1 --port 8765
```

Puis ouvrir:

```text
http://127.0.0.1:8765
```

## Tester

```bash
cd /home/wildlama/agent-gui
python3 -m unittest discover -s tests -v
```

Smoke-test API:

```bash
curl http://127.0.0.1:8765/health
curl -X POST http://127.0.0.1:8765/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"agent":"echo","message":"hello"}'
```

## Ajouter un agent

Éditer:

```text
/home/wildlama/agent-gui/configs/agents.json
```

Exemple:

```json
{
  "id": "mon-agent",
  "name": "Mon Agent",
  "type": "cli",
  "description": "Agent CLI custom",
  "command": ["mon-agent", "run", "{prompt}"],
  "timeout_seconds": 300,
  "enabled": true
}
```

Règles:

- `{prompt}` est remplacé par le message utilisateur.
- `{session_id}` peut être utilisé si l'agent accepte un identifiant de session.
- La commande est une liste d'arguments, pas une string shell.
- Le backend n'utilise pas `shell=True`, pour éviter les injections shell.

## Agents préconfigurés

- `echo`: intégré, marche toujours.
- `hermes`: `hermes chat -q {prompt}`.
- `claude-code`: `claude -p --output-format text --permission-mode default {prompt}`.
- `openclaw`: exemple désactivé, à ajuster selon ton installation.

## Limites V1

- Pas encore de streaming token par token.
- Pas encore d'approbations interactives dans l'UI.
- Les agents CLI sont appelés en one-shot.
- Les sorties tool calls ne sont pas encore parsées profondément, elles sont affichées comme messages ou status.

## Suite logique

- Ajouter SSE ou WebSocket pour streaming temps réel.
- Ajouter des sprites 2D par état à la place de l'ASCII.
- Ajouter un protocole d'événements JSON optionnel pour agents compatibles.
- Ajouter un adaptateur Hermes profond via API gateway ou ACP.
- Ajouter un panneau fichiers / workspace.
- Ajouter approvals UI pour commandes dangereuses.

Voir aussi:

```text
/home/wildlama/agent-gui/PLAN.md
```


1	from __future__ import annotations
2
3	from dataclasses import dataclass
4
5
6	@dataclass(frozen=True)
7	class AvatarFrame:
8	    state: str
9	    label: str
10	    art: str
11
12
13	_FRAMES: dict[str, AvatarFrame] = {
14	    "idle": AvatarFrame(
15	        state="idle",
16	        label="En attente",
17	        art='''
18	        .-"""-.
19	       /  . .  \\
20	       |   ^   |
21	       | \\___/ |
22	        \\_____/
23	       /|  |  |\\
24	      /_|__|__|_\\
25	         /_\\
26	        '''.strip("\n"),
27	    ),
28	    "thinking": AvatarFrame(
29	        state="thinking",
30	        label="Réflexion",
31	        art='''
32	        .-"""-.
33	       /  o o  \\    ?
34	       |   ~   |  ?
35	       |  ___  |
36	        \\_____/
37	       /|  |  |\\
38	      /_|__|__|_\\
39	         /_\\
40	        '''.strip("\n"),
41	    ),
42	    "tool": AvatarFrame(
43	        state="tool",
44	        label="Outil en cours",
45	        art='''
46	        .-"""-.
47	       /  > <  \\   [#]
48	       |   -   |  /|\\
49	       | \\___/ |   |
50	        \\_____/
51	       /|  |  |\\
52	      /_|__|__|_\\
53	         /_\\
54	        '''.strip("\n"),
55	    ),
56	    "speaking": AvatarFrame(
57	        state="speaking",
58	        label="Réponse",
59	        art='''
60	        .-"""-.
61	       /  ^ ^  \\  )))
62	       |   O   |
63	       | \\___/ |
64	        \\_____/
65	       /|  |  |\\
66	      /_|__|__|_\\
67	         /_\\
68	        '''.strip("\n"),
69	    ),
70	    "error": AvatarFrame(
71	        state="error",
72	        label="Erreur",
73	        art='''
74	        .-"""-.
75	       /  x x  \\   !
76	       |   _   |  !!!
77	       | /___\\ |
78	        \\_____/
79	       /|  |  |\\
80	      /_|__|__|_\\
81	         /_\\
82	        '''.strip("\n"),
83	    ),
84	}
85
86
87	def get_avatar(state: str = "idle") -> dict[str, str]:
88	    frame = _FRAMES.get(state, _FRAMES["idle"])
89	    return {"state": frame.state, "label": frame.label, "art": frame.art}
90
91
92	def available_states() -> list[str]:
93	    return list(_FRAMES.keys())
94

from __future__ import annotations

from dataclasses import dataclass

@dataclass(frozen=True)
class AvatarFrame:
    state: str
    label: str
    art: str

_FRAMES: dict[str, AvatarFrame] = {
    "idle": AvatarFrame(
        state="idle",
        label="En attente",
        art='''
        .-"""-.
       /  . .  \\
       |   ^   |
       | \\___/ |
        \\_____/
       /|  |  |\\
      /_|__|__|_\\
         /_\\
        '''.strip("\n"),
    ),
    "thinking": AvatarFrame(
        state="thinking",
        label="Réflexion",
        art='''
        .-"""-.
       /  o o  \\    ?
       |   ~   |  ?
       |  ___  |
        \\_____/
       /|  |  |\\
      /_|__|__|_\\
         /_\\
        '''.strip("\n"),
    ),
    "tool": AvatarFrame(
        state="tool",
        label="Outil en cours",
        art='''
        .-"""-.
       /  > <  \\   [#]
       |   -   |  /|\\
       | \\___/ |   |
        \\_____/
       /|  |  |\\
      /_|__|__|_\\
         /_\\
        '''.strip("\n"),
    ),
    "speaking": AvatarFrame(
        state="speaking",
        label="Réponse",
        art='''
        .-"""-.
       /  ^ ^  \\  )))
       |   O   |
       | \\___/ |
        \\_____/
       /|  |  |\\
      /_|__|__|_\\
         /_\\
        '''.strip("\n"),
    ),
    "error": AvatarFrame(
        state="error",
        label="Erreur",
        art='''
        .-"""-.
       /  x x  \\   !
       |   _   |  !!!
       | /___\\ |
        \\_____/
       /|  |  |\\
      /_|__|__|_\\
         /_\\
        '''.strip("\n"),
    ),
}

def get_avatar(state: str = "idle") -> dict[str, str]:
    frame = _FRAMES.get(state, _FRAMES["idle"])
    return {"state": frame.state, "label": frame.label, "art": frame.art}

def available_states() -> list[str]:
    return list(_FRAMES.keys())


1	<!doctype html>
2
3
4	  <meta charset="utf-8" />
5	  <meta name="viewport" content="width=device-width, initial-scale=1" />
6	  <title>Agent GUI Local</title>
7	  <link rel="stylesheet" href="/styles.css" />
8
9
10

11

12
13

14
AG

15

16
Pixel command center

17
Agent GUI

18
Surcouche locale universelle

19

20

21
22	      Agent actif
23	      <select id="agentSelect"></select>
24

25
26

27	        <button id="newSessionBtn">Nouvelle session</button>
28	        <button id="reloadBtn" class="secondary">Reload crew</button>
29

30
31

32
Sessions

33

34

35
36
37
38

39

40

41
Virtual office

42
Bureau animé

43

44

45	            idle
46	            Session
47

48

49	        <canvas id="officeCanvas" width="960" height="540"></canvas>
50
Les agents arrivent au bureau.

51

52
53

54

55

56
Conversation

57
Chat

58

59
Ctrl+Enter pour envoyer

60

61
62

63
64	        <form id="chatForm" class="composer">
65	          <textarea id="messageInput" rows="3" placeholder="Écris à l'agent..."></textarea>
66	          <button type="submit">Envoyer</button>
67	        </form>
68

69
70
71
72

73
Crew

74
Agents présents

75

76

77
78

79
Selected

80
Agent

81
idle

82

83

84
85

86
Timeline

87
Activity

88

89

90
91

92
93
94
95
96

<!doctype html>

  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <title>Agent GUI Local</title>
  <link rel="stylesheet" href="/styles.css" />

AG

Pixel command center

Agent GUI

Surcouche locale universelle

      Agent actif
      <select id="agentSelect"></select>

        <button id="newSessionBtn">Nouvelle session</button>
        <button id="reloadBtn" class="secondary">Reload crew</button>

Sessions

Virtual office

Bureau animé

            idle
            Session

        <canvas id="officeCanvas" width="960" height="540"></canvas>

Les agents arrivent au bureau.

Conversation

Chat

Ctrl+Enter pour envoyer

        <form id="chatForm" class="composer">
          <textarea id="messageInput" rows="3" placeholder="Écris à l'agent..."></textarea>
          <button type="submit">Envoyer</button>
        </form>

Crew

Agents présents

Selected

Agent

idle

Timeline

Activity


/mnt/DATA/WORK/agent-gui ici je build une GUI pour mes agent IA, pour qu'ils aient un avatar et que je puisse les modeler comme je veux. est ce qu'on pourrait mettre ça dessus? comme ça les avatars po…


Je lis le reste pendant que l'agent finit sa recherche.


1	# Agent GUI Local - Plan de construction
2
3	## Vision
4
5	Créer une web app locale qui sert de surcouche graphique universelle pour des agents LLM existants.
6
7	Objectif: ne pas remplacer Hermes, Claude Code, OpenClaw ou autres agents, mais leur fournir une interface commune avec chat, historique, timeline d'activité, logs, approbations futures, et avatar visuel. La première version utilise un avatar ASCII, puis pourra évoluer vers sprites 2D.
8
9	## Principes produit
10
11	- Local-first: l'app tourne sur `127.0.0.1`, sans cloud obligatoire.
12	- Agent-agnostic: chaque agent est branché via un adaptateur.
13	- Simple à installer: Python stdlib pour le backend, HTML/CSS/JS vanilla pour le frontend.
14	- Extensible: ajout d'un agent via un fichier de config ou une nouvelle classe adapter.
15	- Lisible pour reprise par d'autres agents: code commenté, README, tests, plan de continuité.
16
17	## Architecture prévue
18
19	```text
20	agent-gui/
21	  agent_gui/
22	    __init__.py
23	    app.py              # serveur HTTP local, API REST, endpoints statiques
24	    adapters.py         # protocole commun + adaptateurs CLI
25	    sessions.py         # stockage léger en JSONL
26	    config.py           # chargement configs d'agents
27	    avatar.py           # états et frames ASCII
28	  static/
29	    index.html          # UI principale
30	    styles.css          # style app + avatar ASCII
31	    app.js              # logique frontend, appels API
32	  configs/
33	    agents.json         # agents disponibles: echo, hermes, claude, openclaw
34	  tests/
35	    test_adapters.py
36	    test_sessions.py
37	    test_avatar.py
38	  README.md
39	  PLAN.md
40	```
41
42	## Protocole interne commun
43
44	Chaque agent externe doit être ramené à des événements standardisés:
45
46	```json
47	{
48	  "type": "assistant_message | user_message | tool_request | tool_result | status | error",
49	  "agent": "hermes",
50	  "session_id": "...",
51	  "content": "texte visible",
52	  "raw": "sortie brute optionnelle",
53	  "created_at": "ISO timestamp"
54	}
55	```
56
57	La V1 simplifie: elle exécute un agent CLI en mode one-shot, récupère stdout/stderr, puis retourne un message assistant + événements status.
58
59	## Adaptateurs V1
60
61	### EchoAdapter
62
63	Adaptateur de test intégré. Il renvoie le prompt utilisateur sans appeler de modèle. Il sert aux tests et au smoke-test sans credentials.
64
65	### CliAdapter
66
67	Adaptateur universel pour commandes terminal.
68
69	Exemples prévus dans `configs/agents.json`:
70
71	- Hermes:
72	  - commande: `hermes chat -q {prompt}`
73	- Claude Code:
74	  - commande indicative: `claude -p {prompt}`
75	- OpenClaw:
76	  - commande indicative à ajuster selon installation
77
78	Important: l'adaptateur ne doit jamais passer le prompt par interpolation shell non protégée. Il utilisera une liste d'arguments avec placeholder `{prompt}`.
79
80	## UI V1
81
82	Écran unique:
83
84	- Sidebar agents
85	- Sélecteur d'agent
86	- Chat central
87	- Zone de saisie
88	- Avatar ASCII à droite
89	- Panneau timeline / logs
90	- Boutons:
91	  - envoyer
92	  - nouvelle session
93	  - recharger agents
94
95	États avatar:
96
97	- idle: repos
98	- thinking: réflexion
99	- tool: utilisation outil / commande
100	- speaking: réponse reçue
101	- error: erreur
102
103	## Tests prévus
104
105	- Avatar: chaque état retourne une frame non vide.
106	- Config: chargement du fichier agents JSON.
107	- Adapters:
108	  - EchoAdapter renvoie bien un événement assistant.
109	  - CliAdapter exécute une commande de test Python simple.
110	- Sessions:
111	  - append événement JSONL
112	  - relire historique
113
114	Commandes de validation:
115
116	```bash
117	cd /home/wildlama/agent-gui
118	python3 -m unittest discover -s tests -v
119	python3 -m agent_gui.app --host 127.0.0.1 --port 8765
120	curl http://127.0.0.1:8765/health
121	curl -X POST http://127.0.0.1:8765/api/chat \
122	  -H 'Content-Type: application/json' \
123	  -d '{"agent":"echo","message":"hello"}'
124	```
125
126	## État actuel du travail
127
128	Débuté le 2026-06-15.
129
130	Fait:
131
132	- Dossier projet créé: `/home/wildlama/agent-gui`
133	- Package Python créé: `/home/wildlama/agent-gui/agent_gui/__init__.py`
134	- Plan de reprise créé: `/home/wildlama/agent-gui/PLAN.md`
135	- Backend Python stdlib créé: `agent_gui/app.py`
136	- Adaptateurs créés: `agent_gui/adapters.py`
137	- Config agents créée: `agent_gui/config.py` + `configs/agents.json`
138	- Stockage sessions JSONL créé: `agent_gui/sessions.py`
139	- Avatar ASCII à états créé: `agent_gui/avatar.py`
140	- Frontend créé: `static/index.html`, `static/styles.css`, `static/app.js`
141	- Tests créés: `tests/test_avatar.py`, `tests/test_adapters.py`, `tests/test_config.py`, `tests/test_sessions.py`
142	- README créé: `/home/wildlama/agent-gui/README.md`
143	- Tests unitaires passés: `python3 -m unittest discover -s tests -v` -> 10 tests OK
144	- Serveur lancé en arrière-plan sur `http://127.0.0.1:8765`
145	- Smoke-test API OK: `/health`, `/api/agents`, `/api/chat` avec agent `echo`
146	- Smoke-test navigateur OK: message envoyé via l'UI, réponse Echo affichée
147	- Hermes CLI détecté sur cette machine: `/home/wildlama/.local/bin/hermes`
148
149	En cours:
150
151	- L'interface a été transformée en bureau pixel art avec sprites animés et ambiance arcade.
152	- Rien de bloquant. La V2 visuelle est en place.
153
154	À faire ensuite si un autre agent reprend:
155
156	1. Ajouter streaming temps réel via SSE ou WebSocket.
157	2. Ajouter parsing d'événements JSON structurés pour agents compatibles.
158	3. Ajouter approbations interactives côté UI.
159	4. Ajouter sprites 2D à la place de l'avatar ASCII.
160	5. Créer un adaptateur Hermes profond, au-delà du simple `hermes chat -q`.
161	6. Créer un adaptateur Claude Code profond si son format stream/JSON est disponible.
162	7. Ajouter packaging desktop Tauri si besoin.
163
164	## Notes pour un agent qui reprendrait le travail
165
166	- Ne pas demander plus de specs: partir sur une V1 minimaliste mais fonctionnelle.
167	- Ne pas installer de dépendances lourdes pour la V1.
168	- Préférer Python stdlib pour que ça marche immédiatement.
169	- Ne pas faire une simple maquette statique: il faut une vraie app qui lance au moins l'adaptateur echo et peut appeler des commandes CLI configurées.
170	- Pour les commandes externes, éviter `shell=True`.
171	- Prévoir que Hermes existe probablement sur cette machine, mais ne pas rendre les tests dépendants de Hermes.
172	- Le README doit expliquer comment ajouter un nouvel agent.
173
174	## Roadmap après V1
175
176	- Streaming temps réel via SSE ou WebSocket.
177	- Approbations de commandes dans l'UI.
178	- Import/export sessions.
179	- Avatars sprites 2D par état.
180	- Adaptateur Hermes plus profond via gateway API ou session store.
181	- Adaptateur Claude Code plus profond si son format JSON/stream est disponible.
182	- Thèmes visuels par agent.
183

# Agent GUI Local - Plan de construction

## Vision

Créer une web app locale qui sert de surcouche graphique universelle pour des agents LLM existants.

Objectif: ne pas remplacer Hermes, Claude Code, OpenClaw ou autres agents, mais leur fournir une interface commune avec chat, historique, timeline d'activité, logs, approbations futures, et avatar visuel. La première version utilise un avatar ASCII, puis pourra évoluer vers sprites 2D.

## Principes produit

- Local-first: l'app tourne sur `127.0.0.1`, sans cloud obligatoire.
- Agent-agnostic: chaque agent est branché via un adaptateur.
- Simple à installer: Python stdlib pour le backend, HTML/CSS/JS vanilla pour le frontend.
- Extensible: ajout d'un agent via un fichier de config ou une nouvelle classe adapter.
- Lisible pour reprise par d'autres agents: code commenté, README, tests, plan de continuité.

## Architecture prévue

```text
agent-gui/
  agent_gui/
    __init__.py
    app.py              # serveur HTTP local, API REST, endpoints statiques
    adapters.py         # protocole commun + adaptateurs CLI
    sessions.py         # stockage léger en JSONL
    config.py           # chargement configs d'agents
    avatar.py           # états et frames ASCII
  static/
    index.html          # UI principale
    styles.css          # style app + avatar ASCII
    app.js              # logique frontend, appels API
  configs/
    agents.json         # agents disponibles: echo, hermes, claude, openclaw
  tests/
    test_adapters.py
    test_sessions.py
    test_avatar.py
  README.md
  PLAN.md
```

## Protocole interne commun

Chaque agent externe doit être ramené à des événements standardisés:

```json
{
  "type": "assistant_message | user_message | tool_request | tool_result | status | error",
  "agent": "hermes",
  "session_id": "...",
  "content": "texte visible",
  "raw": "sortie brute optionnelle",
  "created_at": "ISO timestamp"
}
```

La V1 simplifie: elle exécute un agent CLI en mode one-shot, récupère stdout/stderr, puis retourne un message assistant + événements status.

## Adaptateurs V1

### EchoAdapter

Adaptateur de test intégré. Il renvoie le prompt utilisateur sans appeler de modèle. Il sert aux tests et au smoke-test sans credentials.

### CliAdapter

Adaptateur universel pour commandes terminal.

Exemples prévus dans `configs/agents.json`:

- Hermes:
  - commande: `hermes chat -q {prompt}`
- Claude Code:
  - commande indicative: `claude -p {prompt}`
- OpenClaw:
  - commande indicative à ajuster selon installation

Important: l'adaptateur ne doit jamais passer le prompt par interpolation shell non protégée. Il utilisera une liste d'arguments avec placeholder `{prompt}`.

## UI V1

Écran unique:

- Sidebar agents
- Sélecteur d'agent
- Chat central
- Zone de saisie
- Avatar ASCII à droite
- Panneau timeline / logs
- Boutons:
  - envoyer
  - nouvelle session
  - recharger agents

États avatar:

- idle: repos
- thinking: réflexion
- tool: utilisation outil / commande
- speaking: réponse reçue
- error: erreur

## Tests prévus

- Avatar: chaque état retourne une frame non vide.
- Config: chargement du fichier agents JSON.
- Adapters:
  - EchoAdapter renvoie bien un événement assistant.
  - CliAdapter exécute une commande de test Python simple.
- Sessions:
  - append événement JSONL
  - relire historique

Commandes de validation:

```bash
cd /home/wildlama/agent-gui
python3 -m unittest discover -s tests -v
python3 -m agent_gui.app --host 127.0.0.1 --port 8765
curl http://127.0.0.1:8765/health
curl -X POST http://127.0.0.1:8765/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"agent":"echo","message":"hello"}'
```

## État actuel du travail

Débuté le 2026-06-15.

Fait:

- Dossier projet créé: `/home/wildlama/agent-gui`
- Package Python créé: `/home/wildlama/agent-gui/agent_gui/__init__.py`
- Plan de reprise créé: `/home/wildlama/agent-gui/PLAN.md`
- Backend Python stdlib créé: `agent_gui/app.py`
- Adaptateurs créés: `agent_gui/adapters.py`
- Config agents créée: `agent_gui/config.py` + `configs/agents.json`
- Stockage sessions JSONL créé: `agent_gui/sessions.py`
- Avatar ASCII à états créé: `agent_gui/avatar.py`
- Frontend créé: `static/index.html`, `static/styles.css`, `static/app.js`
- Tests créés: `tests/test_avatar.py`, `tests/test_adapters.py`, `tests/test_config.py`, `tests/test_sessions.py`
- README créé: `/home/wildlama/agent-gui/README.md`
- Tests unitaires passés: `python3 -m unittest discover -s tests -v` -> 10 tests OK
- Serveur lancé en arrière-plan sur `http://127.0.0.1:8765`
- Smoke-test API OK: `/health`, `/api/agents`, `/api/chat` avec agent `echo`
- Smoke-test navigateur OK: message envoyé via l'UI, réponse Echo affichée
- Hermes CLI détecté sur cette machine: `/home/wildlama/.local/bin/hermes`

En cours:

- L'interface a été transformée en bureau pixel art avec sprites animés et ambiance arcade.
- Rien de bloquant. La V2 visuelle est en place.

À faire ensuite si un autre agent reprend:

1. Ajouter streaming temps réel via SSE ou WebSocket.
2. Ajouter parsing d'événements JSON structurés pour agents compatibles.
3. Ajouter approbations interactives côté UI.
4. Ajouter sprites 2D à la place de l'avatar ASCII.
5. Créer un adaptateur Hermes profond, au-delà du simple `hermes chat -q`.
6. Créer un adaptateur Claude Code profond si son format stream/JSON est disponible.
7. Ajouter packaging desktop Tauri si besoin.

## Notes pour un agent qui reprendrait le travail

- Ne pas demander plus de specs: partir sur une V1 minimaliste mais fonctionnelle.
- Ne pas installer de dépendances lourdes pour la V1.
- Préférer Python stdlib pour que ça marche immédiatement.
- Ne pas faire une simple maquette statique: il faut une vraie app qui lance au moins l'adaptateur echo et peut appeler des commandes CLI configurées.
- Pour les commandes externes, éviter `shell=True`.
- Prévoir que Hermes existe probablement sur cette machine, mais ne pas rendre les tests dépendants de Hermes.
- Le README doit expliquer comment ajouter un nouvel agent.

## Roadmap après V1

- Streaming temps réel via SSE ou WebSocket.
- Approbations de commandes dans l'UI.
- Import/export sessions.
- Avatars sprites 2D par état.
- Adaptateur Hermes plus profond via gateway API ou session store.
- Adaptateur Claude Code plus profond si son format JSON/stream est disponible.
- Thèmes visuels par agent.


1	const state = {
2	  sessionId: localStorage.getItem('agentGuiSession') || null,
3	  agents: [],
4	  crew: [],
5	  selectedAgentId: null,
6	  sceneMessage: 'Les agents arrivent au bureau.',
7	  selectedStatus: 'idle',
8	};
9
10	const OFFICE = {
11	  baseWidth: 960,
12	  baseHeight: 540,
13	};
14
15	const $ = (id) => document.getElementById(id);
16
17	function clamp(value, min, max) {
18	  return Math.max(min, Math.min(max, value));
19	}
20
21	function lerp(a, b, t) {
22	  return a + (b - a) * t;
23	}
24
25	function hashString(text) {
26	  let hash = 2166136261;
27	  for (let i = 0; i < text.length; i += 1) {
28	    hash ^= text.charCodeAt(i);
29	    hash = Math.imul(hash, 16777619);
30	  }
31	  return hash >>> 0;
32	}
33
34	function colorFromHash(text, lightness = 58, saturation = 78) {
35	  const h = hashString(text) % 360;
36	  return `hsl(${h} ${saturation}% ${lightness}%)`;
37	}
38
39	function paletteFor(id) {
40	  const seed = hashString(id);
41	  const hue = seed % 360;
42	  return {
43	    shirt: `hsl(${hue} 80% 58%)`,
44	    shirtDark: `hsl(${hue} 72% 42%)`,
45	    accent: `hsl(${(hue + 42) % 360} 80% 64%)`,
46	    pants: `hsl(${(hue + 210) % 360} 30% 30%)`,
47	    skin: '#f3c6a4',
48	    hair: '#2a2436',
49	    outline: '#10131b',
50	    light: '#ffffff',
51	  };
52	}
53
54	async function api(path, options = {}) {
55	  const response = await fetch(path, {
56	    headers: { 'Content-Type': 'application/json', ...(options.headers || {}) },
57	    ...options,
58	  });
59	  const data = await response.json();
60	  if (!response.ok) throw new Error(data.error || `HTTP ${response.status}`);
61	  return data;
62	}
63
64	function setStatus(name) {
65	  const pill = $('statusPill');
66	  pill.textContent = name;
67	  pill.className = `pill ${name}`;
68	  state.selectedStatus = name;
69	  const member = currentCrewMember();
70	  if (member) member.mood = name;
71	}
72
73	function setOfficeCaption(text) {
74	  state.sceneMessage = text;
75	  $('officeCaption').textContent = text;
76	}
77
78	function currentCrewMember() {
79	  return state.crew.find((member) => member.id === state.selectedAgentId) || state.crew[0] || null;
80	}

const state = {
  sessionId: localStorage.getItem('agentGuiSession') || null,
  agents: [],
  crew: [],
  selectedAgentId: null,
  sceneMessage: 'Les agents arrivent au bureau.',
  selectedStatus: 'idle',
};

const OFFICE = {
  baseWidth: 960,
  baseHeight: 540,
};

const $ = (id) => document.getElementById(id);

function clamp(value, min, max) {
  return Math.max(min, Math.min(max, value));
}

function lerp(a, b, t) {
  return a + (b - a) * t;
}

function hashString(text) {
  let hash = 2166136261;
  for (let i = 0; i < text.length; i += 1) {
    hash ^= text.charCodeAt(i);
    hash = Math.imul(hash, 16777619);
  }
  return hash >>> 0;
}

function colorFromHash(text, lightness = 58, saturation = 78) {
  const h = hashString(text) % 360;
  return `hsl(${h} ${saturation}% ${lightness}%)`;
}

function paletteFor(id) {
  const seed = hashString(id);
  const hue = seed % 360;
  return {
    shirt: `hsl(${hue} 80% 58%)`,
    shirtDark: `hsl(${hue} 72% 42%)`,
    accent: `hsl(${(hue + 42) % 360} 80% 64%)`,
    pants: `hsl(${(hue + 210) % 360} 30% 30%)`,
    skin: '#f3c6a4',
    hair: '#2a2436',
    outline: '#10131b',
    light: '#ffffff',
  };
}

async function api(path, options = {}) {
  const response = await fetch(path, {
    headers: { 'Content-Type': 'application/json', ...(options.headers || {}) },
    ...options,
  });
  const data = await response.json();
  if (!response.ok) throw new Error(data.error || `HTTP ${response.status}`);
  return data;
}

function setStatus(name) {
  const pill = $('statusPill');
  pill.textContent = name;
  pill.className = `pill ${name}`;
  state.selectedStatus = name;
  const member = currentCrewMember();
  if (member) member.mood = name;
}

function setOfficeCaption(text) {
  state.sceneMessage = text;
  $('officeCaption').textContent = text;
}

function currentCrewMember() {
  return state.crew.find((member) => member.id === state.selectedAgentId) || state.crew[0] || null;
}


J'ai une bonne vision du projet. En attendant le résultat de la recherche LAM, voici ce que je vois déjà :

---

**Ton projet actuel** — stack Python stdlib + JS vanilla, avatar ASCII multi-états, canvas pixel-art animé. La roadmap mentionne explicitement "sprites 2D" comme prochaine étape.

**Ce que LAM fait** (d'après ce que je connais déjà) — Large Animation Model génère un avatar 3D parlant depuis une photo portrait + audio, avec lip-sync et expressions faciales. Output : mesh 3D animé ou vidéo rendue.

**L'intégration est faisable, mais en deux couches distinctes :**

```
[Agent répond] → [TTS → fichier audio] → [LAM → animation 3D/vidéo] → [Affichage canvas]
```

Les vrais points de friction à vérifier :
1. **Latence** — LAM est un modèle lourd (GPU requis), ça va être offline/asynchrone, pas du temps réel
2. **Format de sortie** — si c'est un `.glb`/`.obj` animé, on peut le lire avec Three.js dans le canvas ; si c'est une vidéo MP4, encore plus simple
3. **API Python** — s'il y a une API, on branche ça comme un nouvel adaptateur dans `avatar.py`

Je récupère les détails précis dès que l'agent revient. Une chose est sûre : LAM s'intègrerait comme un **renderer d'avatar optionnel**, en parallèle de l'ASCII existant, sans casser l'architecture.


<task-notification>
<task-id>ac89a0f264ece3ae0</task-id>
<tool-use-id>toolu_019HqPzuqEXi1FDySjwe4Bvh</tool-use-id>
<output-file>/tmp/claude-1000/-home-wildlama/7de9b929-9050-430b-9b36-f74849c1c0a7/tasks/ac89a0f264ece3ae0.output</output-file>
<status>completed</status>
<summary>Agent "Research LAM GitHub repo capabilities" completed</summary>
<result>Here is a concise integration-focused breakdown of LAM:

---

**1. What LAM Actually Does**

LAM stands for **Large Avatar Model** (not "Animation Model"). It reconstructs a **photorealistic, animatable 3D Gaussian Splatting head avatar from a single input photo in ~1.4 seconds**. A companion model called **Audio2Expression** (released April 2025) then drives that avatar with audio input to produce lip-synced facial animation. The project also integrates with **OpenAvatarChat** for real-time AI chatting avatar use cases.

---

**2. Inputs**

- **Avatar creation**: one static photograph (single image)
- **Animation**: a motion sequence file (passed via `${MOTION_SEQ}` argument) — likely facial landmark / expression data
- **Audio-driven animation**: the Audio2Expression module takes audio input (format not specified in README; likely WAV/MP3) and drives the avatar's face

There is no text-to-avatar or video input mentioned.

---

**3. Outputs**

- **3D Gaussian Splatting representation** (not a traditional mesh like GLB/FBX/OBJ — this is a point-cloud-like volumetric format)
- **Real-time rendering**: 562.9 FPS on A100 GPU, 110+ FPS on a mobile phone (Xiaomi 14)
- **ZIP export** compatible with OpenAvatarChat for deployment in interactive chat apps
- No traditional video file output mentioned; the pipeline is designed for real-time streaming/rendering

---

**4. System Requirements**

- **GPU**: NVIDIA GPU required; tested on A100. CUDA 12.1 or 11.8 (Linux scripts provided); CUDA 12.8 one-click Windows installer also available
- **Python**: Not pinned in README; `requirements.txt` exists in the repo
- **VRAM/RAM**: Not documented explicitly
- **Platforms**: Linux (primary), Windows (one-click installer), mobile rendering supported via a separate cross-platform path

---

**5. Python API / Web Demo**

- **No clean Python import API** — usage is CLI (`sh ./scripts/inference.sh`) or Gradio web UI (`python app_lam.py`)
- **Gradio local GUI**: `python app_lam.py`
- **Hosted demos**: HuggingFace Space at `spaces/3DAIGC/LAM`, ModelScope studio, and OpenAvatarChat HuggingFace/ModelScope spaces

---

**6. Integration into a Web/Desktop GUI App**

Feasible but with caveats:

- The Gradio app (`app_lam.py`) is the intended local GUI — you could wrap or extend it
- For deeper integration, you'd need to call the inference script or reverse-engineer it into importable Python functions, since there's no published SDK
- The Gaussian Splatting output format is **not standard 3D** (no GLB/FBX), so rendering in a typical WebGL/three.js or Unity app requires a Gaussian Splatting renderer
- Real-time performance is excellent once the avatar is built, making it viable for live chat/streaming apps via OpenAvatarChat
- A desktop app wrapper around the Gradio interface (e.g., Electron + localhost) is the path of least resistance

---

**7. License**

**Apache License 2.0** — permissive, allows commercial use with attribution. No model weight license restrictions called out in the README beyond this.

---

**Integration Feasibility Summary**

LAM is feasible to integrate into a chatbot/avatar application, especially via the OpenAvatarChat pipeline it was designed for. The main friction points are: (1) no clean Python API — you'd work at the script/CLI level or fork the Gradio app; (2) the Gaussian Splatting output requires a specialized renderer, not a standard 3D engine; (3) hardware requirements are non-trivial (CUDA GPU needed for inference, though rendering works on mobile). If your goal is a talking-head AI avatar app, the Audio2Expression + LAM + OpenAvatarChat stack is the intended path.</result>
<usage><subagent_tokens>15372</subagent_tokens><tool_uses>12</tool_uses><duration_ms>197795</duration_ms></usage>
</task-notification>


Voici le bilan complet.

---

## LAM — ce que c'est vraiment

**Large Avatar Model** (pas Animation) : génère un avatar 3D photoréaliste depuis **une seule photo** en ~1.4 secondes. Le module compagnon **Audio2Expression** anime ensuite cet avatar avec de l'audio pour un lip-sync en temps réel.

### Points clés pour ton projet

| Critère | Réalité |
|---|---|
| **Output** | 3D Gaussian Splatting — **pas** du GLB/FBX standard |
| **Rendu** | 562 FPS sur A100, 110+ FPS sur mobile — très rapide une fois l'avatar généré |
| **API Python** | Aucune propre — CLI (`inference.sh`) ou Gradio local (`app_lam.py`) |
| **GPU requis** | CUDA obligatoire (testé A100) |
| **Licence** | Apache 2.0 — libre |
| **Intégration prévue** | OpenAvatarChat — exactement un use-case "avatar qui parle dans une app de chat" |

---

## Faisabilité sur ton agent-gui

**Oui, c'est intégrable** — mais il y a deux frictions réelles :

**1. Le format Gaussian Splatting n'est pas du WebGL classique**
Three.js ne lit pas ça nativement. Il faut une lib JS dédiée (ex: [gsplat.js](https://github.com/antimatter15/splat)) dans ton canvas. C'est faisable mais non-trivial.

**2. Pas d'API Python propre**
Il faut soit appeler les scripts CLI en subprocess, soit forker le code Gradio pour l'importer comme module.

---

## Architecture d'intégration recommandée

```
[Agent répond] → [TTS → WAV] → [Audio2Expression] → [rendu Gaussian Splatting]
                                       ↓
                              canvas HTML (via gsplat.js)
```

**La voie la plus rapide** : lancer `app_lam.py` (Gradio) en subprocess et l'afficher dans un `<iframe>` dans la sidebar — pas élégant mais opérationnel rapidement.

**La voie propre** : remplacer ton canvas pixel-art par un renderer Gaussian Splatting, et brancher Audio2Expression via un endpoint Python local ajouté à `app.py`.

---

## Ma recommandation

Avant de toucher au code, deux questions pratiques :
1. **Tu as un GPU NVIDIA avec CUDA ?** Sans ça, LAM ne tourne pas du tout.
2. **Tu veux du live (avatar qui parle en temps réel pendant que l'agent répond) ou du pré-rendu (vidéo générée après la réponse) ?**

Si tu as le GPU et que tu veux du temps réel, **OpenAvatarChat** est le chemin directement prévu pour ce cas — on pourrait l'embarquer ou s'y connecter depuis ton GUI. Si tu veux rester léger, on peut d'abord ajouter **TTS + lèvres animées 2D** (bien plus simple) comme étape intermédiaire vers LAM.


j'ai un GPU Nvdia avec cuda ( peut être faut il installer les drivers) je veux du live


Vérifions d'abord ce qu'on a sur ta machine.


Bash

nvidia-smi 2>&1; echo "---"; nvcc --version 2>&1; echo "---"; python3 -c "import torch; print('PyTorch:', torch.__version__); print('CUDA:', torch.cuda.is_available(), torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'N/A')" 2>&1

Check GPU, CUDA, and PyTorch availability


Exit code 1
Mon Jun 15 22:48:54 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.71.05              Driver Version: 595.71.05      CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5090        Off |   00000000:01:00.0  On |                  N/A |
|  0%   51C    P8             38W /  575W |     822MiB /  32607MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            7078      G   /usr/bin/gnome-shell                    211MiB |
|    0   N/A  N/A            7421      G   /usr/bin/Xwayland                        12MiB |
|    0   N/A  N/A            8760    C+G   /usr/bin/nautilus                        74MiB |
|    0   N/A  N/A            8851    C+G   /usr/bin/ptyxis                          52MiB |
|    0   N/A  N/A           10162    C+G   ...rack-uuid=3190708988185955192        284MiB |
+-----------------------------------------------------------------------------------------+
---
/bin/bash: line 1: nvcc: command not found
---
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import torch; print('PyTorch:', torch.__version__); print('CUDA:', torch.cuda.is_available(), torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'N/A')
    ^^^^^^^^^^^^
ModuleNotFoundError: No module named 'torch'


Parfait — **RTX 5090 avec 32 Go VRAM**, c'est largement au-dessus de ce qu'il faut. Les drivers sont là (595.71), CUDA 13.2 supporté. Il manque juste `nvcc` (toolkit) et PyTorch. Vérifions Python.


python3 --version; pip3 --version 2>&1; which conda 2>&1; ls /usr/local/cuda* 2>&1; dpkg -l | grep cuda | head -10

Check Python, pip, conda, and CUDA toolkit install state


Python 3.14.4
/bin/bash: line 1: pip3: command not found
ls: cannot access '/usr/local/cuda*': No such file or directory


pip --version 2>&1; python3 -m pip --version 2>&1; apt list --installed 2>/dev/null | grep -E "python3-pip|cuda" | head -20; ls /opt/ 2>&1

Check pip and installed packages


/bin/bash: line 1: pip: command not found
/usr/bin/python3: No module named pip
google


Check OS and architecture


PRETTY_NAME="Ubuntu 26.04 LTS"
NAME="Ubuntu"
VERSION_ID="26.04"
VERSION="26.04 (Resolute Raccoon)"
VERSION_CODENAME=resolute
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=resolute
LOGO=ubuntu-logo
x86_64


Voilà le tableau complet :

| Élément | État |
|---|---|
| GPU | RTX 5090, 32 Go VRAM ✅ |
| Drivers NVIDIA | 595.71 ✅ |
| CUDA Toolkit (`nvcc`) | Absent ❌ |
| Python | 3.14 (trop récent pour PyTorch) ❌ |
| pip | Absent ❌ |
| conda | Absent ❌ |

**Le vrai problème** : Python 3.14 n'est pas encore supporté par PyTorch ou LAM. Il faut un env isolé Python 3.11. La voie la plus propre est **Miniconda**.

Voici le plan d'installation complet, étape par étape. Chaque bloc `!commande` tu peux le taper directement dans ce chat.

---

## Étape 1 — Installer Miniconda

```bash
! wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O /tmp/miniconda.sh && bash /tmp/miniconda.sh -b -p $HOME/miniconda3
```

Puis active conda pour ta session :

```bash
! $HOME/miniconda3/bin/conda init bash && source ~/.bashrc
```

---

## Étape 2 — Créer un env Python 3.11 avec CUDA

```bash
! $HOME/miniconda3/bin/conda create -n lam python=3.11 -y
```

---

## Étape 3 — Installer CUDA Toolkit 12.8

Pour le RTX 5090 (architecture Blackwell), il faut CUDA 12.8 minimum :

```bash
! $HOME/miniconda3/bin/conda run -n lam conda install -c nvidia cuda-toolkit=12.8 -y
```

---

## Étape 4 — Installer PyTorch 2.7+ (support Blackwell)

```bash
! $HOME/miniconda3/bin/conda run -n lam pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
```

Vérifie que ça marche :

```bash
! $HOME/miniconda3/bin/conda run -n lam python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"
```

---

Une fois PyTorch confirmé, on clône LAM + OpenAvatarChat et on intègre le stream dans ton agent-gui. Lance l'étape 1 et dis-moi ce que tu obtiens.


On installe la stack pour intégrer LAM (avatars 3D parlants en temps réel) dans ton agent-gui. Tu dois lancer l'étape 1 : télécharger et installer Miniconda. (disable recaps in /config)


wildlama@wildlama-MS-7E07:~$ ! $HOME/miniconda3/bin/conda init bash && source ~/.bashrc
bash: /home/wildlama/miniconda3/bin/conda: No such file or directory
wildlama@wildlama-MS-7E07:~$ $HOME/miniconda3/bin/conda init bash && source ~/.bashrc
bash: /home/wildlama/miniconda3/bin/conda: No such file or directory
wildlama@wildlama-MS-7E07:~$


L'installation Miniconda n'a pas été faite — le `wget` n'a pas tourné. Faites les deux commandes dans l'ordre :


ls /tmp/miniconda.sh 2>&1 && ls $HOME/miniconda3 2>&1

Check if miniconda installer or directory exists


Error: Exit code 2
/tmp/miniconda.sh
ls: cannot access '/home/wildlama/miniconda3': No such file or directory


Le fichier est déjà téléchargé dans `/tmp`. Il faut juste lancer l'installeur. Tape ça dans ton terminal :

```bash
bash /tmp/miniconda.sh -b -p $HOME/miniconda3 && $HOME/miniconda3/bin/conda init bash && source ~/.bashrc
```

Le `-b` = mode silencieux sans questions. Si ça plante, colle-moi l'erreur complète.


Installing Miniconda to get Python 3.11 and PyTorch working for LAM (talking 3D avatar on RTX 5090). Run: `bash /tmp/miniconda.sh -b -p $HOME/miniconda3 && $HOME/miniconda3/bin/conda init bash && source ~/.bashrc` (disable recaps in /config)


(base) wildlama@wildlama-MS-7E07:~$ $HOME/miniconda3/bin/conda run -n lam python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"
2.11.0+cu128
True
NVIDIA GeForce RTX 5090
(base) wildlama@wildlama-MS-7E07:~$


PyTorch 2.11 + CUDA 12.8 + RTX 5090 — tout fonctionne. On peut attaquer LAM.

Clône le repo et installe les dépendances :

```bash
cd /mnt/DATA/WORK && git clone https://github.com/aigc3d/LAM.git && cd LAM && $HOME/miniconda3/bin/conda run -n lam pip install -r requirements.txt
```

Ça va prendre quelques minutes. Lance ça et colle-moi la fin de l'output (surtout les erreurs éventuelles).

En parallèle, je regarde la structure du repo pour préparer l'intégration avec ton agent-gui.


Agent

Research LAM repo structure and OpenAvatarChat integration

Prompt:
I need to understand the LAM repository (https://github.com/aigc3d/LAM) and OpenAvatarChat integration for a real-time talking avatar setup. Please fetch and read:

1. The LAM repo README (already known: uses Gaussian Splatting, has app_lam.py Gradio app, Audio2Expression module)
2. Look for the OpenAvatarChat repo at https://github.com/aigc3d/OpenAvatarChat - read its README carefully
3. Understand the pipeline: how does audio go in, how does the rendered output come out (WebSocket? HTTP stream? Video frames?)
4. Look for any Python API or importable functions (not just CLI scripts)
5. Look for the Audio2Expression module specifically - what does it accept as input format?

I want to integrate this into a local web app (Python backend + vanilla JS frontend). The goal is:
- User sends text to AI agent → agent responds → TTS generates audio → LAM animates avatar in real-time → rendered in browser canvas

Report:
- The exact pipeline for real-time audio-driven avatar
- How OpenAvatarChat handles the browser rendering (WebRTC? WebSocket? iframe?)
- Whether there's a Python callable API for inference
- Key files/classes to look at for integration


Auto mode lets Claude handle permission prompts automatically — Claude checks each tool call for risky actions and prompt injection before executing. Actions Claude identifies as safe are executed, while actions Claude identifies as risky are blocked and Claude may try a different approach. Ideal for long-running tasks. Sessions are slightly more expensive. Claude can make mistakes that allow harmful commands to run, it's recommended to only use in isolated environments. Shift+Tab to change mode.


I now have all the information needed for a comprehensive answer. Let me compile the full picture.

Here is a comprehensive technical report on the LAM + OpenAvatarChat pipeline for real-time talking avatar integration.

---

## Pipeline Architecture Overview

The full stack has three distinct layers: **Audio2Expression** (Python, server-side), **LAM Gaussian Splatting Renderer** (either Python server-side render *or* WebGL client-side), and a **WebRTC/WebSocket frontend**.

---

## 1. Real-Time Audio-Driven Avatar Pipeline

The end-to-end flow in OpenAvatarChat (`HumanAIGC-Engineering/OpenAvatarChat`, not `aigc3d/OpenAvatarChat`) is:

```
Browser audio (mic)
  → WebRTC → Python server
    → VAD (Silero) → ASR (SenseVoice) → LLM → TTS (CosyVoice)
      → Audio bytes → LAM Audio2Expression → ARKit blendshapes [52 floats/frame @ 30fps]
        → WebSocket to browser → gaussian-splat-renderer-for-lam (WebGL)
          → Canvas render
```

For your TTS→Avatar use case, the VAD/ASR/LLM modules are bypassed, and TTS audio feeds directly into Audio2Expression.

---

## 2. Audio2Expression Module (`aigc3d/LAM_Audio2Expression`)

**Key repo:** `https://github.com/aigc3d/LAM_Audio2Expression`

**Input format:** Raw NumPy float32 audio at **16 kHz mono**, fed in 1-second chunks (16,000 samples per call).

**Python callable API:**
```python
from engines.defaults import default_argument_parser, default_config_parser, default_setup
from engines.infer import INFER

cfg = default_config_parser('configs/lam_audio2exp_config_streaming.py', options)
cfg = default_setup(cfg)
infer = INFER.build(dict(type=cfg.infer.type, cfg=cfg))
infer.model.eval()

context = None
# For each 1-second audio chunk:
output, context = infer.infer_streaming_audio(audio_chunk_np, sample_rate=16000, context)
# output['expression'] → np.ndarray shape [N_frames, 52]  (30fps ARKit blendshapes)
```

**Streaming context structure:**
```python
context = {
    'is_initial_input': bool,
    'previous_audio': np.ndarray | None,
    'previous_expression': np.ndarray | None,
    'previous_volume': np.ndarray | None,
    'first_input_flag': bool
}
```

**Output:** `[N_frames, 52]` NumPy array. Each frame = 52 ARKit blendshape weights (0.0–1.0). The 52 names are the standard Apple ARKit face channels: `browDownLeft`, `browDownRight`, `browInnerUp`, `cheekPuff`, `eyeBlinkLeft`, `jawOpen`, `mouthSmileLeft`, `tongueOut`, etc.

**Required models:** `wav2vec2-base-960h` (audio encoder) + `LAM_audio2exp` weights (as `LAM_audio2exp.tar` — **note: as of early 2026 this tarball is not yet publicly released**, per open issue #91 in the LAM repo).

---

## 3. Browser Rendering: WebGL + WebSocket (not WebRTC for avatar frames)

**Key repo:** `https://github.com/aigc3d/LAM_WebRender`

The browser rendering is done via the NPM package `gaussian-splat-renderer-for-lam`. It is **WebGL-based Gaussian Splatting**, not video streaming. The avatar geometry (Gaussian Splat model) is loaded once as a ZIP file; only the expression data streams per frame.

**JavaScript API:**
```typescript
import * as GaussianSplats3D from "gaussian-splat-renderer-for-lam";

const renderer = await GaussianSplats3D.GaussianSplatRenderer.getInstance(
  divElement,          // HTMLDivElement canvas container
  "./avatar.zip",      // Pre-exported LAM avatar (GLB+Gaussian data)
  {
    getChatState: () => "Responding",  // "Idle"|"Listening"|"Thinking"|"Responding"
    getExpressionData: () => ({        // Called every frame; return dict of 52 blendshape name→value
      jawOpen: 0.4,
      mouthSmileLeft: 0.2,
      // ...all 52 ARKit channels
    }),
    backgroundColor: "0x000000",
    alpha: 0.0
  }
);
```

The renderer **polls `getExpressionData()` each animation frame** — there is no WebSocket push from server to renderer. This means your Python backend feeds blendshapes to a JS bridge (your own WebSocket), and the JS side passes them into `getExpressionData()` on each rAF tick.

**Avatar asset format:** A ZIP file containing the LAM-exported GLB (Gaussian Splat geometry baked from a source photo). Generated via `app_lam.py --blender_path /path/to/blender` with the "Export ZIP for Chatting Avatar" checkbox enabled. Requires Blender 4.0+ and the FBX SDK.

---

## 4. OpenAvatarChat WebRTC: for Audio I/O Only

**The WebRTC layer** (`gradio-webrtc`) is used *only for browser microphone audio input and TTS audio output back to the browser* — not for streaming avatar frames. The avatar render happens entirely in WebGL client-side. The LAM client handler is a **WebSocket** (`ws_lam_client_handler`) that delivers blendshape data from Python to the browser's JavaScript renderer.

---

## 5. LAM Core Model: Python-Importable Inference

The LAM 3D Gaussian model itself (`lam/models/modeling_lam.py`) is callable programmatically:

```python
from lam.runners.infer.lam import LAMInferrer

inferrer = LAMInferrer()
# Takes: source image + pre-computed FLAME params (from motion sequences)
# Returns: rendered RGB frames [Nv, H, W, 3]
```

**FLAME parameter keys** (`gs_renderer.py`):
- `expr`: [B, Nv, 100] — expression coefficients
- `betas`: [B, 100] — shape (constant per identity)
- `rotation`, `neck_pose`, `jaw_pose`, `eyes_pose`, `translation` — pose per frame
- `teeth_bs` (optional) — teeth blendshapes

However, for your use case you **do not need to call the LAM Gaussian renderer in Python at all** — the WebGL renderer does the rendering client-side from the pre-exported ZIP. The Python side only needs Audio2Expression.

---

## 6. Key Files to Study for Integration

| File | Repo | Purpose |
|---|---|---|
| `engines/infer.py` | `aigc3d/LAM_Audio2Expression` | `infer_streaming_audio()` — core callable API |
| `models/network.py` | `aigc3d/LAM_Audio2Expression` | `Audio2Expression` model (Wav2Vec2 → 52 blendshapes) |
| `models/utils.py` | `aigc3d/LAM_Audio2Expression` | `ARKitBlendShape` list + `export_blendshape_animation()` |
| `inference_streaming_audio.py` | `aigc3d/LAM_Audio2Expression` | Reference streaming usage |
| `src/gaussianAvatar.ts` | `aigc3d/LAM_WebRender` | JS renderer API and expression data protocol |
| `handlers/avatar/lam/avatar_handler_lam_audio2expression.py` | `HumanAIGC-Engineering/OpenAvatarChat` | How audio bytes → blendshapes → WebSocket works in production |
| `config/chat_with_lam.yaml` | `HumanAIGC-Engineering/OpenAvatarChat` | Full handler stack configuration |
| `app_lam.py` | `aigc3d/LAM` | Avatar export (ZIP) with Blender |

---

## 7. Integration Plan for Your Use Case

Your pipeline (TTS audio → animated avatar in browser canvas):

1. **Python backend**: Load `Audio2ExpressionInfer` once at startup. For each TTS response, convert audio to 16kHz numpy, call `infer_streaming_audio()` in 1-second chunks, buffer output blendshape frames.
2. **WebSocket**: Stream blendshape frames as JSON `{jawOpen: 0.4, eyeBlinkLeft: 0.1, ...}` to the browser at 30fps.
3. **Browser JS**: Load `gaussian-splat-renderer-for-lam` with the pre-exported avatar ZIP. Wire `getExpressionData()` to read from the WebSocket buffer. The renderer calls this each animation frame and deforms the Gaussian Splat mesh live in WebGL.

**Critical blocker to verify:** The `LAM_audio2exp` model weights (`LAM_audio2exp.tar`) appear to not be publicly available as of February 2026 (GitHub issue #91 in `aigc3d/LAM` is open and unanswered). You may need to contact the maintainers directly or monitor the repo for a release.

agentId: a8905ecd71f2dd288 (use SendMessage with to: 'a8905ecd71f2dd288' to continue this agent)
<usage>subagent_tokens: 72637
tool_uses: 142
duration_ms: 759572</usage>

Prompt:
I need to understand the LAM repository (https://github.com/aigc3d/LAM) and OpenAvatarChat integration for a real-time talking avatar setup. Please fetch and read:

1. The LAM repo README (already known: uses Gaussian Splatting, has app_lam.py Gradio app, Audio2Expression module)
2. Look for the OpenAvatarChat repo at https://github.com/aigc3d/OpenAvatarChat - read its README carefully
3. Understand the pipeline: how does audio go in, how does the rendered output come out (WebSocket? HTTP stream? Video frames?)
4. Look for any Python API or importable functions (not just CLI scripts)
5. Look for the Audio2Expression module specifically - what does it accept as input format?

I want to integrate this into a local web app (Python backend + vanilla JS frontend). The goal is:
- User sends text to AI agent → agent responds → TTS generates audio → LAM animates avatar in real-time → rendered in browser canvas

Report:
- The exact pipeline for real-time audio-driven avatar
- How OpenAvatarChat handles the browser rendering (WebRTC? WebSocket? iframe?)
- Whether there's a Python callable API for inference
- Key files/classes to look at for integration


(base) wildlama@wildlama-MS-7E07:~$ $HOME/miniconda3/bin/conda run -n lam python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))" 2.1…


Voilà l'architecture complète — c'est très clair maintenant.

---

## Pipeline exact pour du live

```
[Agent répond] → [TTS → audio 16kHz mono]
    → [Python: Audio2Expression → 52 blendshapes ARKit @ 30fps]
        → [WebSocket JSON] → [Browser: gaussian-splat-renderer-for-lam WebGL]
                                    ↑
                              Avatar ZIP (pré-exporté depuis ta photo)
```

**Ce qui est élégant** : le renderer WebGL tourne côté browser, Python n'envoie que des tableaux de 52 floats par frame — très léger.

---

## Un blocage potentiel à vérifier

Le modèle `LAM_audio2exp.tar` (les poids d'Audio2Expression) n'était **pas encore publié** début 2026 (issue #91 du repo LAM ouverte). On est en juin 2026, ça a peut-être changé. Vérifie ici :
```
https://github.com/aigc3d/LAM/issues/91
```

---

## Ce qu'il faut faire dans l'ordre

**Pendant que `pip install -r requirements.txt` tourne**, dis-moi :

1. Est-ce que le pip install LAM s'est bien terminé ?
2. Est-ce que tu as **Blender 4.0+** installé ? (nécessaire pour exporter l'avatar ZIP depuis une photo)

Ensuite les étapes seront :

1. Clôner `LAM_Audio2Expression` + vérifier que les poids sont disponibles
2. Exporter ton avatar ZIP via `app_lam.py` (une photo → ton avatar 3D)
3. Ajouter un **WebSocket** dans `agent_gui/app.py`
4. Ajouter le renderer WebGL (`gaussian-splat-renderer-for-lam`) dans ton canvas HTML


Installing LAM on your RTX 5090 to add a real-time 3D talking avatar to agent-gui. Check if the pip install finished and whether Blender 4.0+ is installed. (disable recaps in /config)


je n'en sais rien, comment je sais ca si le pip install s'est terminé ?


Je vérifie directement.


Bash

$HOME/miniconda3/bin/conda run -n lam pip show torch 2>&1 | head -5; echo "---"; ls /mnt/DATA/WORK/LAM 2>&1 | head -20

Check if LAM repo exists and torch is installed in lam env


Name: torch
Version: 2.11.0+cu128
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org
Author:
---
ls: cannot access '/mnt/DATA/WORK/LAM': No such file or directory


Le repo LAM n'a pas encore été cloné. Je le fais maintenant.


cd /mnt/DATA/WORK && git clone https://github.com/aigc3d/LAM.git 2>&1

Clone LAM repository


Cloning into 'LAM'...
Shell cwd was reset to /home/wildlama


ToolSearch

Question: select:ExitPlanMode


Question: select:ExitPlanMode


Bash

Check if LAM was cloned


LICENSE
LICENSE_WEIGHT
README.md
README_CN.md
app_hf_space.py
app_lam.py
assets
configs
external
lam
requirements.txt
scripts
tools
vhap


LAM est cloné. Je lance l'install des dépendances — ça peut prendre 5-10 min.


cd /mnt/DATA/WORK/LAM && $HOME/miniconda3/bin/conda run -n lam pip install -r requirements.txt 2>&1 | tail -20

Install LAM requirements in lam conda env


        File "/tmp/pip-build-env-twk3z2jm/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 301, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-twk3z2jm/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 520, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-twk3z2jm/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 317, in run_setup
          exec(code, locals())
        File "<string>", line 15, in <module>
      ModuleNotFoundError: No module named 'torch'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed to build 'git+https://github.com/facebookresearch/pytorch3d.git' when getting requirements to build wheel
ERROR conda.cli.main_run:execute(148): `conda run pip install -r requirements.txt` failed. (See above for error)
Collecting git+https://github.com/facebookresearch/pytorch3d.git (from -r requirements.txt (line 26))
  Cloning https://github.com/facebookresearch/pytorch3d.git to /tmp/pip-req-build-qtfphu_r
  Resolved https://github.com/facebookresearch/pytorch3d.git to commit 7f8a8a142fa1383a7895ed01dd7082105f9a81fb
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
Shell cwd was reset to /home/wildlama


Read LAM requirements


pymcubes==0.1.6
omegaconf==2.3.0
moviepy==1.0.3
pandas==2.2.3
transformers==4.41.2
numpy==1.23.0
trimesh==4.4.9
opencv_python_headless==4.11.0.86
tensorflow==2.12.0
face-detection-tflite==0.6.0
scikit-image==0.20.0
jaxlib==0.4.30
gradio==3.44.3
huggingface_hub==0.23.2
Cython
accelerate
tyro
einops
diffusers
plyfile
jaxtyping
typeguard
chumpy
loguru
ninja
git+https://github.com/facebookresearch/pytorch3d.git
git+https://github.com/ashawkey/diff-gaussian-rasterization/
nvdiffrast@git+https://github.com/ShenhanQian/nvdiffrast@backface-culling
git+https://github.com/camenduru/simple-knn/


Le problème : `pytorch3d` compile depuis les sources et ne trouve pas torch pendant la build. Il faut installer dans le bon ordre.


cd /mnt/DATA/WORK/LAM && $HOME/miniconda3/bin/conda run -n lam pip install pymcubes==0.1.6 omegaconf==2.3.0 moviepy==1.0.3 pandas==2.2.3 transformers==4.41.2 trimesh==4.4.9 opencv_python_headless==4.11.0.86 scikit-image==0.20.0 gradio==3.44.3 huggingface_hub==0.23.2 Cython accelerate tyro einops diffusers plyfile jaxtyping typeguard chumpy loguru ninja 2>&1 | tail -5

Install LAM non-GPU requirements first


  Downloading chumpy-0.70.tar.gz (50 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
Shell cwd was reset to /home/wildlama


cd /mnt/DATA/WORK/LAM && $HOME/miniconda3/bin/conda run -n lam pip install pymcubes==0.1.6 omegaconf==2.3.0 moviepy==1.0.3 pandas==2.2.3 transformers==4.41.2 trimesh==4.4.9 opencv_python_headless==4.11.0.86 scikit-image==0.20.0 gradio==3.44.3 huggingface_hub==0.23.2 Cython accelerate tyro einops diffusers plyfile jaxtyping typeguard loguru ninja 2>&1 | tail -5

Install requirements without chumpy


    Found existing installation: MarkupSafe 3.0.3
    Uninstalling MarkupSafe-3.0.3:
      Successfully uninstalled MarkupSafe-3.0.3

Successfully installed Cython-3.2.5 PyWavelets-1.9.0 PyYAML-6.0.3 accelerate-1.14.0 aiofiles-23.2.1 altair-5.5.0 annotated-doc-0.0.4 annotated-types-0.7.0 antlr4-python3-runtime-4.9.3 anyio-4.13.0 attrs-26.1.0 certifi-2026.5.20 charset_normalizer-3.4.7 click-8.4.1 contourpy-1.3.3 cycler-0.12.1 decorator-4.4.2 diffusers-0.32.2 docstring-parser-0.18.0 einops-0.8.2 fastapi-0.137.1 ffmpy-1.0.0 fonttools-4.63.0 gradio-3.44.3 gradio-client-0.5.0 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 huggingface_hub-0.23.2 idna-3.18 imageio-2.37.3 imageio_ffmpeg-0.6.0 importlib-resources-6.5.2 importlib_metadata-9.0.0 jaxtyping-0.3.11 jsonschema-4.26.0 jsonschema-specifications-2025.9.1 kiwisolver-1.5.0 lazy_loader-0.5 loguru-0.7.3 markupsafe-2.1.5 matplotlib-3.11.0 moviepy-1.0.3 narwhals-2.22.1 ninja-1.13.0 numpy-1.26.4 omegaconf-2.3.0 opencv_python_headless-4.11.0.86 orjson-3.11.9 pandas-2.2.3 pillow-10.4.0 plyfile-1.1.3 proglog-0.1.12 psutil-7.2.2 pydantic-2.13.4 pydantic-core-2.46.4 pydub-0.25.1 pymcubes-0.1.6 pyparsing-3.3.2 python-dateutil-2.9.0.post0 python-multipart-0.0.32 pytz-2026.2 referencing-0.37.0 regex-2026.5.9 requests-2.34.2 rpds-py-2026.5.1 safetensors-0.8.0 scikit-image-0.20.0 scipy-1.17.1 semantic-version-2.10.0 six-1.17.0 starlette-1.3.1 tifffile-2026.3.3 tokenizers-0.19.1 tqdm-4.68.2 transformers-4.41.2 trimesh-4.4.9 typeguard-4.5.2 typing-inspection-0.4.2 tyro-1.0.13 tzdata-2026.2 urllib3-2.7.0 uvicorn-0.49.0 wadler-lindig-0.1.7 websockets-11.0.3 zipp-4.1.0
Shell cwd was reset to /home/wildlama


Base installée. Maintenant pytorch3d avec `--no-build-isolation` pour qu'il trouve torch :


$HOME/miniconda3/bin/conda run -n lam pip install --no-build-isolation "git+https://github.com/facebookresearch/pytorch3d.git" 2>&1 | tail -10

Install pytorch3d with no-build-isolation


Downloading portalocker-3.2.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: pytorch3d, iopath
  Building wheel for pytorch3d (pyproject.toml): started
  Building wheel for pytorch3d (pyproject.toml): finished with status 'error'
  Building wheel for iopath (pyproject.toml): started
  Building wheel for iopath (pyproject.toml): finished with status 'done'
  Created wheel for iopath: filename=iopath-0.1.10-py3-none-any.whl size=31533 sha256=ddbf7263ac8a79bc2c5145090b16d70a49e4774e026cd6acecf5c85c88c0f518
  Stored in directory: /home/wildlama/.cache/pip/wheels/ba/5e/16/6117f8fe7e9c0c161a795e10d94645ebcf301ccbd01f66d8ec
Successfully built iopath
Failed to build pytorch3d


$HOME/miniconda3/bin/conda run -n lam pip install --no-build-isolation "git+https://github.com/facebookresearch/pytorch3d.git" 2>&1 | grep -A5 "error\|Error" | head -30

Get pytorch3d build error details


  error: subprocess-exited-with-error

  × Building wheel for pytorch3d (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [296 lines of output]
      <string>:109: UserWarning: The environment variable `CUB_HOME` was not found. NVIDIA CUB is required for compilation and can be downloaded from `https://github.com/NVIDIA/cub/releases`. You can unpack it to a location of your choice and set the environment variable `CUB_HOME` to the folder containing the `CMakeListst.txt` file.
--
          raise RuntimeError(
      RuntimeError: The current installed version of /home/wildlama/miniconda3/envs/lam/bin/x86_64-conda-linux-gnu-c++ (14.3.0) is greater than the maximum required version by CUDA 12.8. Please make sure to use an adequate version of /home/wildlama/miniconda3/envs/lam/bin/x86_64-conda-linux-gnu-c++ (>=6.0.0, <14.0).
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pytorch3d
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> pytorch3d
ERROR conda.cli.main_run:execute(148): `conda run pip install --no-build-isolation git+https://github.com/facebookresearch/pytorch3d.git` failed. (See above for error)
Collecting git+https://github.com/facebookresearch/pytorch3d.git
  Cloning https://github.com/facebookresearch/pytorch3d.git to /tmp/pip-req-build-8q92azyg
  Resolved https://github.com/facebookresearch/pytorch3d.git to commit 7f8a8a142fa1383a7895ed01dd7082105f9a81fb
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
--
  Building wheel for pytorch3d (pyproject.toml): finished with status 'error'
Failed to build pytorch3d


Le compilateur g++ 14.3 est trop récent pour CUDA 12.8. Je règle ça avec un g++ compatible.


g++ --version 2>&1; which g++ 2>&1; ls /usr/bin/g++* 2>&1; apt list --installed 2>/dev/null | grep g++ | head -10

Check available g++ versions on system


g++ (Ubuntu 15.2.0-16ubuntu1) 15.2.0
Copyright (C) 2025 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

/usr/bin/g++
/usr/bin/g++
/usr/bin/g++-15
g++-15-x86-64-linux-gnu/resolute,now 15.2.0-16ubuntu1 amd64 [installed,automatic]
g++-15/resolute,now 15.2.0-16ubuntu1 amd64 [installed,automatic]
g++-x86-64-linux-gnu/resolute,now 4:15.2.0-5ubuntu1 amd64 [installed,automatic]
g++/resolute,now 4:15.2.0-5ubuntu1 amd64 [installed,automatic]


apt-cache show g++-13 2>&1 | head -5; apt-cache show g++-14 2>&1 | head -5

Check if g++-13 or g++-14 is available in apt


Package: g++-13
Source: gcc-13
Priority: optional
Section: universe/devel
Installed-Size: 34
Package: g++-14
Source: gcc-14
Priority: optional
Section: universe/devel
Installed-Size: 42


Le compilateur conda est trop récent (14.3). Je remplace par gcc 13 dans l'env lam.


$HOME/miniconda3/bin/conda install -n lam -c conda-forge gxx=13 gcc=13 -y 2>&1 | tail -5

Install gcc 13 in lam conda env


                                                      done
Preparing transaction: | done
Verifying transaction: - \ done
Executing transaction: / done


$HOME/miniconda3/bin/conda run -n lam x86_64-conda-linux-gnu-c++ --version 2>&1

Verify gcc 13 is now active in lam env


x86_64-conda-linux-gnu-c++ (conda-forge gcc 13.4.0-19) 13.4.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


GCC 13.4 — compatible. Je relance pytorch3d :


$HOME/miniconda3/bin/conda run -n lam pip install --no-build-isolation "git+https://github.com/facebookresearch/pytorch3d.git" 2>&1 | tail -15

Build pytorch3d with gcc 13


  Using cached iopath-0.1.10-py3-none-any.whl
Requirement already satisfied: tqdm in ./miniconda3/envs/lam/lib/python3.11/site-packages (from iopath->pytorch3d==0.7.9) (4.68.2)
Requirement already satisfied: typing-extensions in ./miniconda3/envs/lam/lib/python3.11/site-packages (from iopath->pytorch3d==0.7.9) (4.15.0)
Collecting portalocker (from iopath->pytorch3d==0.7.9)
  Using cached portalocker-3.2.0-py3-none-any.whl.metadata (8.7 kB)
Using cached portalocker-3.2.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: pytorch3d
  Building wheel for pytorch3d (pyproject.toml): started
  Building wheel for pytorch3d (pyproject.toml): finished with status 'done'
  Created wheel for pytorch3d: filename=pytorch3d-0.7.9-cp311-cp311-linux_x86_64.whl size=6027468 sha256=25c5ffddfffb5b490ea5711a1592cd6e5f8148946a8c2ccead98f99175100919
  Stored in directory: /tmp/pip-ephem-wheel-cache-2j3a5h46/wheels/39/02/3b/eab9735f985044755f4e6d9e8473bfb8b68dc63723658e2ac2
Successfully built pytorch3d
Installing collected packages: portalocker, iopath, pytorch3d

Successfully installed iopath-0.1.10 portalocker-3.2.0 pytorch3d-0.7.9


pytorch3d installé. Je finis les derniers packages :


$HOME/miniconda3/bin/conda run -n lam pip install --no-build-isolation \
  "git+https://github.com/ashawkey/diff-gaussian-rasterization/" \
  "nvdiffrast@git+https://github.com/ShenhanQian/nvdiffrast@backface-culling" \
  "git+https://github.com/camenduru/simple-knn/" 2>&1 | tail -10

Install remaining GPU packages with no-build-isolation


  Created wheel for nvdiffrast: filename=nvdiffrast-0.3.3-py3-none-any.whl size=139936 sha256=3d6d539ef1274c2dd6b3d75378cfb07950ae7578faa2a34de78b2cb3c260ad77
  Stored in directory: /tmp/pip-ephem-wheel-cache-dnnwpcxw/wheels/79/8d/d2/f2a2d42a39126b18ad7d88c1a7d5e6d99b55beb9e397f2eb03
  Building wheel for simple_knn (pyproject.toml): started
  Building wheel for simple_knn (pyproject.toml): finished with status 'done'
  Created wheel for simple_knn: filename=simple_knn-1.0.0-cp311-cp311-linux_x86_64.whl size=672056 sha256=e92532926a90821cc4ee4b443a99b5eb786280ff3c052545c739cbac2a444d08
  Stored in directory: /tmp/pip-ephem-wheel-cache-dnnwpcxw/wheels/f9/0f/01/59bd6ffc8ab1c36e2dd0da2e41eb0a7bb6f69b49df67537e02
Successfully built diff_gaussian_rasterization nvdiffrast simple_knn
Installing collected packages: simple_knn, diff_gaussian_rasterization, nvdiffrast

Successfully installed diff_gaussian_rasterization-0.0.0 nvdiffrast-0.3.3 simple_knn-1.0.0


Tout est installé. Maintenant on vérifie que LAM tourne, et on clône Audio2Expression :


cd /mnt/DATA/WORK && git clone https://github.com/aigc3d/LAM_Audio2Expression.git 2>&1 && ls LAM_Audio2Expression/

Clone LAM_Audio2Expression repo


Cloning into 'LAM_Audio2Expression'...
LICENSE
README.md
app_lam_audio2exp.py
assets
configs
engines
inference.py
inference_streaming_audio.py
models
requirements.txt
scripts
utils
wheels
Shell cwd was reset to /home/wildlama


cat /mnt/DATA/WORK/LAM_Audio2Expression/README.md | head -80

Read Audio2Expression README for setup instructions


# LAM-A2E: Audio to Expression

[![Website](https://raw.githubusercontent.com/prs-eth/Marigold/main/doc/badges/badge-website.svg)](https://aigc3d.github.io/projects/LAM/)
[![Apache License](https://img.shields.io/badge/📃-Apache--2.0-929292)](https://www.apache.org/licenses/LICENSE-2.0)
[![ModelScope Demo](https://img.shields.io/badge/%20ModelScope%20-Space-blue)](https://www.modelscope.cn/studios/Damo_XR_Lab/LAM-A2E)

## Description
#### This project leverages audio input to generate ARKit blendshapes-driven facial expressions in ⚡real-time⚡, powering ultra-realistic 3D avatars generated by [LAM](https://github.com/aigc3d/LAM).
To enable ARKit-driven animation of the LAM model, we adapted ARKit blendshapes to align with FLAME's facial topology through manual customization. The LAM-A2E network follows an encoder-decoder architecture, as shown below. We adopt the state-of-the-art pre-trained speech model Wav2Vec for the audio encoder. The features extracted from the raw audio waveform are combined with style features and fed into the decoder, which outputs stylized blendshape coefficients.

## Demo

  <video controls src="https://github.com/user-attachments/assets/a89a0d70-a573-4d61-91bd-4f09a0b6ce2c">
  </video>

## 📢 News

**[May 21, 2025]** We have released a [Avatar Export Feature](https://www.modelscope.cn/studios/Damo_XR_Lab/LAM_Large_Avatar_Model), enabling users to generate facial expressions from audio using any [LAM-generated](https://github.com/aigc3d/LAM) 3D digital humans.

**[April 21, 2025]** We have released the [ModelScope](https://www.modelscope.cn/studios/Damo_XR_Lab/LAM-A2E) Space !

**[April 21, 2025]** We have released the WebGL Interactive Chatting Avatar SDK on [OpenAvatarChat](https://github.com/HumanAIGC-Engineering/OpenAvatarChat) (including LLM, ASR, TTS, Avatar), with which you can freely chat with our generated 3D Digital Human ! 🔥

### To do list
- [ ] Release Huggingface space.
- [x] Release [Modelscope demo space](https://www.modelscope.cn/studios/Damo_XR_Lab/LAM-A2E). You can try the demo or pull the demo source code and deploy it on your own machine.
- [ ] Release the LAM-A2E model based on the Flame expression.
- [x] Release Interactive Chatting Avatar SDK with [OpenAvatarChat](https://www.modelscope.cn/studios/Damo_XR_Lab/LAM-A2E), including LLM, ASR, TTS, LAM-Avatars.

## 🚀 Get Started
### Environment Setup
```bash
git clone git@github.com:aigc3d/LAM_Audio2Expression.git
cd LAM_Audio2Expression
# Create conda environment (currently only supports Python 3.10)
conda create -n lam_a2e python=3.10
# Activate the conda environment
conda activate lam_a2e
# Install with Cuda 12.1
sh  ./scripts/install/install_cu121.sh
# Or Install with Cuda 11.8
sh ./scripts/install/install_cu118.sh
```

### Download

```
# HuggingFace download
# Download Assets and Model Weights
huggingface-cli download 3DAIGC/LAM_audio2exp --local-dir ./
tar -xzvf LAM_audio2exp_assets.tar && rm -f LAM_audio2exp_assets.tar
tar -xzvf LAM_audio2exp_streaming.tar && rm -f LAM_audio2exp_streaming.tar

# Or OSS Download (In case of HuggingFace download failing)
# Download Assets
wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LAM/LAM_audio2exp_assets.tar
tar -xzvf LAM_audio2exp_assets.tar && rm -f LAM_audio2exp_assets.tar
# Download Model Weights
wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LAM/LAM_audio2exp_streaming.tar
tar -xzvf LAM_audio2exp_streaming.tar && rm -f LAM_audio2exp_streaming.tar

Or Modelscope Download
git clone https://www.modelscope.cn/Damo_XR_Lab/LAM_audio2exp.git ./modelscope_download
```

### Quick Start Guide
#### Using Gradio Interface:
We provide a simple Gradio demo with **WebGL Render**, and you can get rendering results by uploading audio in seconds.

[//]: # ()

  <video controls src="https://github.com/user-attachments/assets/2bb4e74f-cd96-4c50-9833-fae10b1ead4c


Les poids du modèle sont disponibles sur HuggingFace. Mais Audio2Expression demande **Python 3.10** — je crée un env séparé.


Read install script to adapt for CUDA 12.8


# install torch 2.1.2
# or conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia
pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu121

# install dependencies
pip install -r requirements.txt

# install H5-render
pip install wheels/gradio_gaussian_render-0.0.3-py3-none-any.whl


Read Audio2Expression requirements


#spleeter==2.4.0
opencv_python_headless==4.11.0.86
gradio==5.25.2
omegaconf==2.3.0
addict==2.4.0
yapf==0.40.1
librosa==0.11.0
transformers==4.36.2
termcolor==3.0.1
numpy==1.26.3
patool


Dépendances légères — pas de pytorch3d. Je crée l'env Python 3.10 et j'installe tout.


$HOME/miniconda3/bin/conda create -n lam_a2e python=3.10 -y 2>&1 | tail -3

Create lam_a2e conda env with Python 3.10


#
#     $ conda deactivate


$HOME/miniconda3/bin/conda run -n lam_a2e pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 2>&1 | tail -5

Install PyTorch cu128 in lam_a2e env


    Found existing installation: setuptools 82.0.1
    Uninstalling setuptools-82.0.1:
      Successfully uninstalled setuptools-82.0.1

Successfully installed MarkupSafe-3.0.3 cuda-bindings-12.9.4 cuda-pathfinder-1.2.2 cuda-toolkit-12.8.1 filelock-3.29.0 fsspec-2026.4.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.4.2 numpy-2.2.6 nvidia-cublas-cu12-12.8.4.1 nvidia-cuda-cupti-cu12-12.8.90 nvidia-cuda-nvrtc-cu12-12.8.93 nvidia-cuda-runtime-cu12-12.8.90 nvidia-cudnn-cu12-9.19.0.56 nvidia-cufft-cu12-11.3.3.83 nvidia-cufile-cu12-1.13.1.3 nvidia-curand-cu12-10.3.9.90 nvidia-cusolver-cu12-11.7.3.90 nvidia-cusparse-cu12-12.5.8.93 nvidia-cusparselt-cu12-0.7.1 nvidia-nccl-cu12-2.28.9 nvidia-nvjitlink-cu12-12.8.93 nvidia-nvshmem-cu12-3.4.5 nvidia-nvtx-cu12-12.8.90 pillow-12.2.0 setuptools-70.2.0 sympy-1.14.0 torch-2.11.0+cu128 torchaudio-2.11.0+cu128 torchvision-0.26.0+cu128 triton-3.6.0 typing-extensions-4.15.0


cd /mnt/DATA/WORK/LAM_Audio2Expression && $HOME/miniconda3/bin/conda run -n lam_a2e pip install -r requirements.txt 2>&1 | tail -5 && $HOME/miniconda3/bin/conda run -n lam_a2e pip install wheels/gradio_gaussian_render-0.0.3-py3-none-any.whl 2>&1 | tail -3

Install Audio2Expression requirements and gradio wheel


    Found existing installation: numpy 2.2.6
    Uninstalling numpy-2.2.6:
      Successfully uninstalled numpy-2.2.6

Successfully installed addict-2.4.0 aiofiles-24.1.0 annotated-doc-0.0.4 annotated-types-0.7.0 antlr4-python3-runtime-4.9.3 anyio-4.13.0 audioread-3.1.0 certifi-2026.5.20 cffi-2.0.0 charset_normalizer-3.4.7 click-8.4.1 decorator-5.3.1 exceptiongroup-1.3.1 fastapi-0.137.1 ffmpy-1.0.0 gradio-5.25.2 gradio-client-1.8.0 groovy-0.1.2 h11-0.16.0 hf-xet-1.5.1 httpcore-1.0.9 httpx-0.28.1 huggingface-hub-0.36.2 idna-3.18 importlib-metadata-9.0.0 joblib-1.5.3 lazy_loader-0.5 librosa-0.11.0 llvmlite-0.47.0 markdown-it-py-4.2.0 mdurl-0.1.2 msgpack-1.2.0 numba-0.65.1 numpy-1.26.3 omegaconf-2.3.0 opencv_python_headless-4.11.0.86 orjson-3.11.9 pandas-2.3.3 patool-4.0.1 pillow-11.3.0 platformdirs-4.10.0 pooch-1.9.0 pycparser-3.0 pydantic-2.11.10 pydantic-core-2.33.2 pydub-0.25.1 pygments-2.20.0 python-dateutil-2.9.0.post0 python-multipart-0.0.32 pytz-2026.2 pyyaml-6.0.3 regex-2026.5.9 requests-2.34.2 rich-15.0.0 ruff-0.15.17 safehttpx-0.1.7 safetensors-0.8.0 scikit-learn-1.7.2 scipy-1.15.3 semantic-version-2.10.0 shellingham-1.5.4 six-1.17.0 soundfile-0.14.0 soxr-1.1.0 starlette-0.52.1 termcolor-3.0.1 threadpoolctl-3.6.0 tokenizers-0.15.2 tomli-2.4.1 tomlkit-0.13.3 tqdm-4.68.2 transformers-4.36.2 typer-0.26.7 typing-inspection-0.4.2 tzdata-2026.2 urllib3-2.7.0 uvicorn-0.49.0 websockets-15.0.1 yapf-0.40.1 zipp-4.1.0
Requirement already satisfied: urllib3<3,>=1.26 in /home/wildlama/miniconda3/envs/lam_a2e/lib/python3.10/site-packages (from requests->huggingface-hub>=0.28.1->gradio<6.0,>=4.0->gradio-gaussian-render==0.0.3) (2.7.0)
Installing collected packages: gradio-gaussian-render
Successfully installed gradio-gaussian-render-0.0.3
Shell cwd was reset to /home/wildlama


Tout installé. Maintenant on télécharge les poids du modèle :


cd /mnt/DATA/WORK/LAM_Audio2Expression && $HOME/miniconda3/bin/conda run -n lam_a2e huggingface-cli download 3DAIGC/LAM_audio2exp --local-dir ./ 2>&1 | tail -15

Download LAM Audio2Expression model weights from HuggingFace


Fetching 4 files:   0%|          | 0/4 [00:00<?, ?it/s]Downloading 'LAM_audio2exp_assets.tar' to '.cache/huggingface/download/xn1GHoL6lVREWtyO9EhaiJhnvqg=.f5759f080ec1352d454d88d5a32f09c0fc8aeaf50b9ac1fd9c34a4acd82acfa1.incomplete'
Downloading 'LAM_audio2exp_streaming.tar' to '.cache/huggingface/download/o0S-c6zjkluMpxZRzHxvu4G9aD0=.0877f419b1b7b6856479b743b71625ad478f97434404e91288f9f0d6d516b97a.incomplete'
Downloading 'README.md' to '.cache/huggingface/download/Xn7B-BWUGOee2Y6hCZtEhtFu4BE=.7b95401dc46245ac339fc25059d4a56d90b4cde5.incomplete'
Downloading '.gitattributes' to '.cache/huggingface/download/wPaCkH-WbT7GsmxMKKrNZTV4nSM=.a6344aac8c09253b3b630fb776ae94478aa0275b.incomplete'
Download complete. Moving file to README.md
Download complete. Moving file to .gitattributes

Fetching 4 files:  25%|██▌       | 1/4 [00:00<00:01,  2.52it/s]Download complete. Moving file to LAM_audio2exp_assets.tar

Fetching 4 files:  50%|█████     | 2/4 [00:02<00:02,  1.18s/it]Download complete. Moving file to LAM_audio2exp_streaming.tar

Fetching 4 files:  75%|███████▌  | 3/4 [00:03<00:01,  1.28s/it]
Fetching 4 files: 100%|██████████| 4/4 [00:03<00:00,  1.13it/s]
⚠  Warning: 'huggingface-cli download' is deprecated. Use 'hf download' instead.
/mnt/DATA/WORK/LAM_Audio2Expression
Shell cwd was reset to /home/wildlama


cd /mnt/DATA/WORK/LAM_Audio2Expression && tar -xzvf LAM_audio2exp_assets.tar && rm -f LAM_audio2exp_assets.tar && tar -xzvf LAM_audio2exp_streaming.tar && rm -f LAM_audio2exp_streaming.tar && ls -la

Extract model weights tarballs


assets/
assets/images/
assets/images/logo.jpeg
assets/images/snapshot.png
assets/images/teaser.jpg
assets/sample_audio/
assets/sample_audio/BarackObama_english.wav
assets/sample_audio/FangXiao_japanese.wav
assets/sample_audio/HillaryClinton_english.wav
assets/sample_audio/LiBai_TTS_chinese.wav
assets/sample_audio/LinJing_TTS_chinese.wav
assets/sample_audio/Nangyanwen_chinese.wav
assets/sample_audio/XitongShi_japanese.wav
assets/sample_input/
assets/sample_input/barbara.jpg
assets/sample_input/james.png
assets/sample_input/messi.png
assets/sample_input/status.png
assets/sample_input/vfhq_case1.png
assets/sample_lam/
assets/sample_lam/barbara/
assets/sample_lam/barbara/arkitWithBSData/
assets/sample_lam/barbara/arkitWithBSData/.DS_Store
assets/sample_lam/barbara/arkitWithBSData/animation.glb
assets/sample_lam/barbara/arkitWithBSData/bsData.json
assets/sample_lam/barbara/arkitWithBSData/offset.ply
assets/sample_lam/barbara/arkitWithBSData/skin.glb
assets/sample_lam/barbara/arkitWithBSData/vertex_order.json
assets/sample_lam/james/
assets/sample_lam/james/arkitWithBSData/
assets/sample_lam/james/arkitWithBSData/animation.glb
assets/sample_lam/james/arkitWithBSData/bsData.json
assets/sample_lam/james/arkitWithBSData/offset.ply
assets/sample_lam/james/arkitWithBSData/skin.glb
assets/sample_lam/james/arkitWithBSData/vertex_order.json
assets/sample_lam/status/
assets/sample_lam/status/arkitWithBSData/
assets/sample_lam/status/arkitWithBSData/animation.glb
assets/sample_lam/status/arkitWithBSData/bsData.json
assets/sample_lam/status/arkitWithBSData/offset.ply
assets/sample_lam/status/arkitWithBSData/skin.glb
assets/sample_lam/status/arkitWithBSData/vertex_order.json
assets/sample_lam/vfhq_case1/
assets/sample_lam/vfhq_case1/.DS_Store
assets/sample_lam/vfhq_case1/arkitWithBSData/
assets/sample_lam/vfhq_case1/arkitWithBSData/animation.glb
assets/sample_lam/vfhq_case1/arkitWithBSData/bsData.json
assets/sample_lam/vfhq_case1/arkitWithBSData/offset.ply
assets/sample_lam/vfhq_case1/arkitWithBSData/skin.glb
assets/sample_lam/vfhq_case1/arkitWithBSData/vertex_order.json
pretrained_models/
pretrained_models/lam_audio2exp_streaming.tar
total 56
drwxrwxr-x+                    1 wildlama wildlama  4096 Jun 15 23:27 .
drwxrwxr-x+                    1 wildlama wildlama     0 Jun 15 23:25 ..
drwxrwxr-x+                    1 wildlama wildlama     0 Jun 15 23:27 .cache
drwxrwxr-x+                    1 wildlama wildlama  4096 Jun 15 23:25 .git
-rw-rw-r--+                    1 wildlama wildlama  1519 Jun 15 23:27 .gitattributes
-rw-rw-r--+                    1 wildlama wildlama   151 Jun 15 23:25 .gitignore
-rw-rw-r--+                    1 wildlama wildlama 11356 Jun 15 23:25 LICENSE
-rw-rw-r--+                    1 wildlama wildlama    31 Jun 15 23:27 README.md
-rw-rw-r--+                    1 wildlama wildlama 11525 Jun 15 23:25 app_lam_audio2exp.py
drwxrwxr-x+                    1 wildlama wildlama     0 Apr 17  2025 assets
drwxrwxr-x+                    1 wildlama wildlama     0 Jun 15 23:25 configs
drwxrwxr-x+                    1 wildlama wildlama  4096 Jun 15 23:25 engines
-rw-rw-r--+                    1 wildlama wildlama  1301 Jun 15 23:25 inference.py
-rw-rw-r--+                    1 wildlama wildlama  1879 Jun 15 23:25 inference_streaming_audio.py
drwxrwxr-x+                    1 wildlama wildlama  4096 Jun 15 23:25 models
drwxrwxr-x+                    1 wildlama wildlama     0 Apr 17  2025 pretrained_models
-rw-rw-r--+                    1 wildlama wildlama   184 Jun 15 23:25 requirements.txt
drwxrwxr-x+                    1 wildlama wildlama     0 Jun 15 23:25 scripts
drwxrwxr-x+                    1 wildlama wildlama  4096 Jun 15 23:25 utils
drwxrwxr-x+                    1 wildlama wildlama     0 Jun 15 23:25 wheels
Shell cwd was reset to /home/wildlama


cd /mnt/DATA/WORK/LAM_Audio2Expression && tar -xzvf pretrained_models/lam_audio2exp_streaming.tar -C pretrained_models/ 2>&1 | head -10 && ls pretrained_models/

Extract streaming model weights


gzip: stdin: invalid compressed data--crc error

gzip: stdin: invalid compressed data--length error
tar: Child returned status 1
tar: Error is not recoverable: exiting now
lam_audio2exp_streaming.tar
Shell cwd was reset to /home/wildlama


ls -lh /mnt/DATA/WORK/LAM_Audio2Expression/pretrained_models/lam_audio2exp_streaming.tar && file /mnt/DATA/WORK/LAM_Audio2Expression/pretrained_models/lam_audio2exp_streaming.tar

Check the tar file size and type


-rwxrwxr-x+ 1 wildlama wildlama 390M Apr 15  2025 /mnt/DATA/WORK/LAM_Audio2Expression/pretrained_models/lam_audio2exp_streaming.tar
/mnt/DATA/WORK/LAM_Audio2Expression/pretrained_models/lam_audio2exp_streaming.tar: Zip archive data, at least v0.0 to extract, compression method=store


C'est un ZIP, pas un gzip. Je l'extrais correctement.


cd /mnt/DATA/WORK/LAM_Audio2Expression/pretrained_models && unzip -o lam_audio2exp_streaming.tar 2>&1 | tail -10 && ls -la

Unzip model weights


 extracting: lam_audio2exp_flow/data/92
 extracting: lam_audio2exp_flow/data/93
 extracting: lam_audio2exp_flow/data/94
 extracting: lam_audio2exp_flow/data/95
 extracting: lam_audio2exp_flow/data/96
 extracting: lam_audio2exp_flow/data/97
 extracting: lam_audio2exp_flow/data/98
 extracting: lam_audio2exp_flow/data/99
 extracting: lam_audio2exp_flow/version
 extracting: lam_audio2exp_flow/.data/serialization_id
total 398968
drwxrwxr-x+    1 wildlama wildlama         0 Jun 15 23:28 .
drwxrwxr-x+    1 wildlama wildlama      4096 Jun 15 23:27 ..
drwxrwxr-x+    1 wildlama wildlama         0 Jun 15 23:28 lam_audio2exp_flow
-rwxrwxr-x+    1 wildlama wildlama 408538564 Apr 15  2025 lam_audio2exp_streaming.tar
Shell cwd was reset to /home/wildlama


Modèle extrait. On teste que ça tourne avec un audio sample :


cd /mnt/DATA/WORK/LAM_Audio2Expression && $HOME/miniconda3/bin/conda run -n lam_a2e python inference_streaming_audio.py 2>&1 | head -40

Test Audio2Expression inference with sample audio