Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: Remove sInfo from activeSessions before unloading (#5938)

Browse Source 
This commit addresses a potential race condition that could lead to "connection errors" when unloading a llamacpp model.

The issue arose because the `activeSessions` map still has the session info of the model during unload. This could lead to "connection errors" when the backend is taking time to unload while there is an ongoing request to the model.

The fix involves:

1. **Deleting the `pid` from `activeSessions` before calling backend's unload:** This ensures that the model is cleared from the map before we start unloading.
2. **Failure handling**: If somehow the backend fails to unload, the session info for that model is added back to prevent any race conditions.

This commit improves the robustness and reliability of the unloading process by preventing potential conflicts.
This commit is contained in:
Akarshan Biswas 2025-07-27 14:37:34 +05:30 committed by GitHub
parent 54d44ce741
commit c9b44eec52
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
extensions/llamacpp-extension/src/index.ts
View File

@ -1264,6 +1264,8 @@ export default class llamacpp_extension extends AIEngine {
}
const pid = sInfo.pid
try {
this.activeSessions.delete(pid)
// Pass the PID as the session_id
const result = await invoke<UnloadResult>('unload_llama_model', {
pid: pid,
@ -1271,15 +1273,16 @@ export default class llamacpp_extension extends AIEngine {
// If successful, remove from active sessions
if (result.success) {
this.activeSessions.delete(pid)
logger.info(`Successfully unloaded model with PID ${pid}`)
} else {
logger.warn(`Failed to unload model: ${result.error}`)
this.activeSessions.set(sInfo.pid, sInfo)
}
return result
} catch (error) {
logger.error('Error in unload command:', error)
this.activeSessions.set(sInfo.pid, sInfo)
return {
success: false,
error: `Failed to unload model: ${error}`,