j<UdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl m Z ddlmZej eZe ej)j*j*Zedz dz Zedz d z Zed z d z d z Zed z dz dz Zed z dz dz ZeefD]Zej;ddgdZded<iddddddddddddddddddd dd!d"d#d"d$d"d%d"d&d"d'd"d(d"d"d"d"d)Z d*ed+<d,d-d.Z!d*ed/<dLd0Z"dMd1Z#dNd2Z$dOd3Z%dOd4Z&dPdQd5Z'ejPZ)da*d6ed7<d8a+dRd9Z,dSd:Z-dTd;Z.dUd<Z/dVd=Z0d>Z1d?Z2e1e2d@dAf dWdBZ3e1e2f dXdCZ4dDZ5dYdEZ6dSdFZ7dZdGZ8dZdHZ9d[dIZ:dOdJZ;d\dKZ.mp4 and are served at /static/cache/video/.mp4 via the static mount in main.py. ) annotationsN)Path)OptionalvendorWav2Lip checkpointszwav2lip_gan.pthstaticassetsavatarsdatacachevideotmpT)parentsexist_ok)female_one.mp4female_two.mp4female_three.mp4female_four.mp4female_five.mp4female_six.mp4female_seven.mp4female_eight.mp4female_nine.mp4female_ten.mp4 male_one.mp4 male_two.mp4male_three.mp4 male_four.mp4 male_five.mp4 male_six.mp4male_seven.mp4male_eight.mp4 male_nine.mp4 male_ten.mp4 list[str]AVATAR_FILENAMESrfemalerrrrrrrrrrmalerrrr r!r")r#r$r%zdict[str, str] AVATAR_GENDERonyxnova)r)r(VOICE_BY_GENDERctjr)tdtjdD}|r|StDcgc]}t|z j r|c}Scc}w)ulReturn the names of avatar files actually present on disk. Discovery is now DYNAMIC — we scan static/assets/avatars/ for any .mp4 file rather than relying on the hardcoded AVATAR_FILENAMES. That means an operator can drop new avatar clips into the folder and they show up in the picker / random pool without a code change (per the "scalable so new avatar videos automatically appear" requirement on the avatar-selection feature). The hardcoded list is still used as a fall-through if the directory is missing or empty, so dev environments without the asset bundle still boot. c3VK|]!}|js|j#ywN)is_filename.0ps p/Users/priyanka/Documents/AI_Agent_Assessment/pepole_hub_candidate/people_hub_candidate/services/video_engine.py z)list_available_avatars..sO'?!199;qvv'?s))*.mp4) AVATAR_DIRis_dirsortedglobr'r1)foundr2s r6list_available_avatarsr>sgOzw'?OO L) )   & & ( )  s A)c|sy|j}|tvr t|S|jdry|jdryy)zInfer a gender label from a filename. Conventional names are `male_*.mp4` / `female_*.mp4`. Anything else falls back to "male" as a conservative default (same fallback as AVATAR_GENDER for unknown keys).r)r()lowerr* startswith)r2r@s r6_gender_from_filenamerBsO  JJLE }T"" !  c g}tD]d}t|z } |jr|jjnd}|j |t||d|t|df|S#t $rd}Y=wxYw)zReturn one dict per available avatar: { name, gender, size_bytes, url } Used by the GET /api/avatars endpoint that powers the candidate- facing avatar picker modal. Names are sorted so the picker order is stable between calls. r/static/assets/avatars/)r2gender size_bytesurlvoice) r>r9r1statst_size ExceptionappendrBvoice_for_avatar)itemsr2pathsizes r6avatar_manifestrRsE&(D  *.,,.499;&&aD  /53D6:*40   ) L D s,A66 BBc.tj|dS)uiReturn 'male' or 'female' for a given avatar filename. Falls back to 'male' for unknown filenames — see the comment on AVATAR_GENDER. The fallback is conservative on purpose: a wrong gender produces the bug we're trying to fix, so we'd rather have an unknown avatar default to a single consistent voice than have it picked at random. r))r*get avatar_names r6 avatar_genderrWs   [& 11rCcNtjt|tdS)aReturn the OpenAI TTS voice ID that matches this avatar's gender. The role-play WebSocket calls this immediately after picking the avatar and threads the result through every stream_tts_chunks() call for the session, so audio and video always agree on gender.r))r-rTrWrUs r6rNrNs!   }[9?6;R SSrCc4txst}|rA|jj}|Dcgc]}t ||k(s|}}|s|}n|}|r&t j |}|j|St j|Scc}w)uMReturn a random avatar filename (e.g. 'male_one.mp4'). Pass ``seed`` (e.g. the session_id) to get a deterministic pick for a given session — the same session always uses the same avatar so the candidate doesn't see the face change mid-conversation. Calling again with the same seed returns the same avatar. Pass ``gender`` ('male' or 'female') to restrict the pool to avatars of that gender. When omitted the full pool is used. This is the seam used by scenarios that want to lock the interviewer's gender (e.g. ``ai_gender`` on the scenario JSON). )r>r'stripr@rWrandomRandomchoice)seedrF availableapoolrngs r6 pick_avatarrcs'(<,L|S{)BT)I%JKL , , --rCcFddl}|j}tj|d5}|j d|j ||j ||j|ddd|jS#1swY|jSxYw)uWrap raw PCM-16-LE bytes in a WAV header. Wav2Lip's audio loader (librosa) needs a real audio container — it can't parse headerless PCM directly.rNwbr) ioBytesIOwaveopen setnchannels setsampwidth setframerate writeframesgetvalue)rrr_iobufwfs r6pcm_to_wav_bytesrsy  ++-C 3    %  $ s  <<>  <<>s ABB i>cKtjd tddddt|dddttd d d d t|g }t j |t jjt jjd d{}|jd{\}}|jdk7rS|xsdjdddd}tjd|j|td|jd|jr|jj dk(r tdy77w)aSResample any WAV/PCM file to 16 kHz mono PCM-16-LE. No-op if the input is already 16 kHz mono; ffmpeg is smart enough to copy in that case. We always go through ffmpeg rather than librosa here because Wav2Lip's own dependency tree already pulls in ffmpeg and we don't want to add another resampler that might disagree. ffmpegNz0ffmpeg is required for Wav2Lip audio resampling.-y-iz-acr{z-ar-c:a pcm_s16lez -loglevelerrorrirjrrCrignorei$z6[video_engine] ffmpeg resample failed rc=%s stderr= %szffmpeg resample failed (exit ).z(ffmpeg resample produced no output file.)rr RuntimeErrorrrWAV2LIP_NATIVE_SAMPLE_RATEasynciocreate_subprocess_execrpr communicaterdecodermrr1rJrK)src_pathdst_pathcmdproc_rjerr_tails r6_resample_wav_to_16krs>||H%MNN$ s8} s s-. W H  C// !!&&!!&& D &&((IAv !Mc))'8  )s%BEEE2E3B#EEcVtjxrtjS)zCheap pre-flight check used by the health endpoint and to fail fast with a clear message rather than a cryptic subprocess error.)ror: WAV2LIP_CKPTr1rrCr6rlrls!     :L$8$8$::rCcKtjd tddddddt|dt|dd dd d d d ddddddddddt|g}tj ddj |tj|tjjtjjdd{}|jd{\}}|jdk7rS|xsdjdddd}tjd |j|td!|jd"|jr|j!j"dk(r td#y77w)$uReliable lip-sync fallback when Wav2Lip can't render. Plays the avatar MP4 (looping if shorter than the audio) while the TTS audio plays as the soundtrack. The mouth movement comes from the original recording rather than being computed against the TTS, so it's not phoneme-accurate — but the avatar's lips/face *do* move while the AI speaks, which is what the user actually wants visually. Crucially this path has no ML dependencies (only ffmpeg, which is required for Wav2Lip too) so it works in every environment where Wav2Lip's Python deps are flaky. The duration is driven by the audio (-shortest) so the talk video ends exactly when the AI stops speaking — same property as the silence-trimmed Wav2Lip output, no trailing animation. rNzDffmpeg is required for the video fallback. Install it on the server.rz -stream_loopz-1rz-mapz0:v:0z1:a:0z-c:vlibx264z-presetveryfastz-pix_fmtyuv420praacz-b:a128kz -shortestz -movflagsz +faststartz&[video_engine] ffmpeg dub fallback: %s rrrCrr0z6[video_engine] ffmpeg dub failed rc=%s stderr_tail= %szffmpeg dub failed (exit rz#ffmpeg dub produced no output file.)rrrrrrmrnjoinrrrprrrrrr1rJrK) face_path audio_pathout_pathrrrrjrs r6_run_ffmpeg_dubr sr ||H%abb $ c)n c*o 9j*ivv\ H  CHH 5sxx}E// !!&&!!&& D &&((IAv !Mc))'8  )s%CFF F F!B#FFc8Kts td|jd} t||d{|}t}|rdnd}|rdnd}dd d ttd t|d t|d t|ddddddddt|dt|g}t jd|||dj|ttj|tttjj tjj dd{} | j#d{\} } | j$dk7rS| xsdj'dddd} t j)d| j$| td| j$d|j+r|j-j.dk(r td y7#t$r!}t j d|Yd}~d}~wwxYw77ԭw)!uEInvoke Wav2Lip's inference.py as a subprocess. Blocks an event-loop thread for the duration of inference (3–8 s on GPU, 30–90 s on CPU) — the caller is expected to hold ``_inference_lock`` so other turns wait their turn. Three sync-quality fixes vs the previous build: 1. Audio is RESAMPLED to 16 kHz mono before Wav2Lip sees it. The mel encoder was trained on 16 kHz; feeding 24 kHz shifted the phoneme→viseme mapping by ~25 ms which read as the avatar's mouth running a beat ahead of the audio. 2. GPU acceleration when CUDA is available (env var WAV2LIP_USE_GPU=true or auto-detected). On GPU we also raise the wav2lip and face_det batches — both single-digit on CPU, double-digit on GPU. 3. Tighter --pads. The previous 5/12/0/0 was tuned for one specific avatar; 0/15/0/0 works better across the full 32-avatar set because the larger bottom pad covers the chin region across different framings without the top pad pulling the mask into the nose. zEWav2Lip is not installed. Run scripts/setup_wav2lip.sh on the server.z.16k.wavNuH[video_engine] audio resample failed (%s) — using original sample rate@rfz inference.pyz--checkpoint_pathz--facez--audioz --outfilez--padsr15z--resize_factorr{z--wav2lip_batch_sizez--face_det_batch_sizezN[video_engine] running wav2lip gpu=%s wav_batch=%d face_batch=%d cmd=%s cwd=%srrgrrCrrrz3[video_engine] wav2lip failed rc=%s stderr_tail= %szWav2Lip inference failed (exit z). See server logs for details.z Wav2Lip produced no output file.)rlr with_suffixrrLrmrtrrrrrnrrorrrprrrrrr1rJrK) rrrresampled_pathrvuse_gpu wav_batch face_batchrrrirjrs r6 _run_wav2liprOs0 ! S   ++J7Ne":~>>># &'GAIAJ .S.S^S_S]S$SSIZ CHHXJ {//  !!&&!!&&  D  ++--NFF !Mc))'8>">c ? e ^`cdd e:  .s^'HG)G&G)C+H+H,HHB"H&G)) H2HHHHHcK|s td|s t}t|z }|jst d|dt |}t ||}t|}|jrG|jjdkDr*tjd|dd|t|||dd Stjd |dd|t|}t|d z }|j|d } t 4d{|jr7|jjdkDrtjd |ddn- t#rt%|||d{n t'd dddd{|j/dt|||d |dS77F#t($r8}tj+d|d}t-|||d{7Yd}~sd}~wwxYw7o#1d{7swYxYw#t($rY{wxYw#|j/dw#t($rYwwxYwxYww)aMain entry point. Returns a dict:: { "url": "/static/cache/video/.mp4", "key": "", "avatar": "male_one.mp4", "cached": True | False, } Caller passes raw 16-bit PCM @ 24 kHz mono (matches what `services.openai_service.stream_tts_chunks` produces). We wrap it in a WAV file, hash, and either reuse the cached video or generate a fresh one. On cache hit the call returns in < 5 ms. zaudio_pcm is emptyzAvatar not found: z5. Place the 4 MP4 files under static/assets/avatars/.rz)[video_engine] cache hit key=%s avatar=%sN T)rHravatarcachedu8[video_engine] cache miss key=%s avatar=%s — renderingz.wavFz0[video_engine] cache filled while waiting key=%sz-Wav2Lip not installed; using ffmpeg fallback.uA[video_engine] wav2lip path unavailable (%s) — using ffmpeg dub) missing_ok)rHrrrfallback) ValueErrorrcr9r1FileNotFoundErrorrrrrJrKrmrnrrTMP_DIR write_bytes_inference_lockrlrrrLrtrunlink) audio_pcmrV avatar_pathrr wav_bytes audio_tmp used_fallbackwav_errs r6get_or_generaters:" -.. !m {*K      .B C  &i0I9k2C"3'Hhmmo559 .s"I[1668#3#3[s$&) wav2lip_installed wav2lip_dircheckpoint_presentffmpeg_on_path avatars_foundavatars_expected avatar_dir cache_dir cache_countcache_size_bytes)rexistslistr<rlrrrorr1rrr>r'r9rsum) cache_filess r6healthr! s4=3C3C3E$y~~g./2K35";/+335%||H5TA57/":"9~";/""I["II  rC)returnr&)r2rrr"rr)r"z list[dict])rVrrr"rr)NN)r^ Optional[str]rFr#r"rr)r"None)r"bool)rbytesrVrrr"rr)rrrr"r)rrrr"rr) rr&rrrrrrrrr"r&)rr&rrrrr"r&)rrrrr"r$)rrrrrrr"r$)rr&rVrrr"dict)r"r')=__doc__ __future__rrrloggingrr[rstructrprpathlibrtypingr getLogger__name__rm__file__resolveparentBASE_DIRrorr9rr_dmkdirr'__annotations__r*r-r>rBrRrWrNrcLockrrdrkrwrrrrTTS_SAMPLE_RATETTS_SAMPLE_WIDTHrrrrrlrrrrr!rrCr6r:s)T#  g!x.((*1188X% 1  -0AA X%09< V#g-7 V#g-5 g BHHTDH) ()(`)!)!)!  )!)!)!)!)!)!"#)!&')!*f+)!.f/)!2f3)!6f7)!:f;)!>f?)!BfC)!FO)! ~)l# 0&0 2T@',,.# ^"$GN H $,.=.>+.-/ +.'*+.(++.&)+.(+ +.5: +.^)8)9 "% #& >C 0$GD; ,B^T?nXv3rC