jtUdZddlZddlZddlZddlZddlZddlZddlmZmZddl m Z ddl m Z ddl mZmZmZmZddlmZddlmZdae eed <dae eed <d efd Zd efd Zd efdZdefdZdededefdZ dUde!dede ede e"d ejFf dZ$ded e%fdZ&dddddZ'd ed e(fd!Z)d ed e(fd"Z*d#d$d%d&ed'ed(ed)ed*ed ef d+Z+d,d-ded(ed.ed/e(d e%f d0Z,d1ed e efd2Z-d3ed4ed e efd5Z.d6ed dfd7Z/d8eded4ed e efd9Z0d8eded4ed e efd:Z1d;ed4ed e efd<Z2d=d>d?d@dAdBZ3dCdDdEdFdGdBZ4dHdIdJddKdLed(edMe%dNedOe%dPed dfdQZ5dRed(ed e(fdSZ6ded dfdTZ7y)Vu GOLS - AI Avatar Video Generation Pipeline ========================================== Runs as a FastAPI BackgroundTask (no Celery needed). Pipeline stages (executed in order): 1. TTS - script -> voice.mp3 (OpenAI TTS -> gTTS -> pyttsx3 -> silence) 2. Base / Liveness - image|video + audio -> base.mp4 • image + GPU models OFF -> FFmpeg "liveness" (Ken-Burns motion + subtle head sway) so the avatar looks alive • video source -> looped/clipped to audio length 3. Face animation (LivePortrait / SadTalker) - run if model installed 4. Lip sync (Wav2Lip) - run if model installed 5. Face enhancement (CodeFormer / GFPGAN) - run if model installed 6. Watermark (logo overlay, opacity/position) - if enabled in settings 7. Thumbnail + save to outputs/ + record in videos.json GPU-only stages (3-5) are *actually invoked* now (previously TODO stubs). Each runner is guarded: if the model directory / entrypoint is missing or the run fails, the pipeline logs it and falls back to the previous stage's output so a video is always produced. On a machine with the models installed and a CUDA GPU, they execute and chain their outputs. N)datetimetimezone)Optional)cfg)projects_store avatars_store videos_storesettings_store)log)openai_service_FFMPEG_FFPROBEreturnc  trtStjxsdj}|rk|dk7rftj j |r|atS|jdr|n|dz}tj j |r|atStjd}|r|atSddddtj jtjjdddd d tj jtjjd ddd d g}|D]A}tj j |s#|atjd |tcStd )Nffmpeg.exezC:\ffmpeg\bin\ffmpeg.exez&C:\Program Files\ffmpeg\bin\ffmpeg.exez,C:\Program Files (x86)\ffmpeg\bin\ffmpeg.exezC:\tools\ffmpeg\bin\ffmpeg.exe USERPROFILEbinz ffmpeg.exe LOCALAPPDATAz[pipeline] Found ffmpeg at: %saffmpeg not found! FIX (Windows): 1. Download from https://www.gyan.dev/ffmpeg/builds/ 2. Extract to C:\ffmpeg\ 3. Add C:\ffmpeg\bin to PATH, OR set FFMPEG_PATH in backend/.env FIX (macOS): brew install ffmpeg FIX (Linux): sudo apt install ffmpeg Then restart the app.)r r FFMPEG_PATHstripospathisfileendswithshutilwhichjoinenvirongetr info RuntimeError)env_pathexefound win_pathsps m/Users/priyanka/Documents/AI_Agent_Assessment/ai-video-generator-gols-v2-new/backend/app/services/pipeline.py _find_ffmpegr*-s9%2,,.HH( 77>>( #GN"++F3hF9J 77>># GN LL "E #17)  RZZ^^M26%V  RZZ^^NB75,W I 77>>! G HH5q 9N     cdtrtSt}tjj tjj |d}t jdk(r|dz }tjj|r|atStjdxsdatS)Nffprobewin32r) rr*rrrdirnamesysplatformrrr) ffmpeg_pathr-s r) _find_ffprober3Ws~.Kggll277??;7CG ||w6 ww~~g O<< *7i Or+cdtjtjj SN)rnowrutc isoformatr+r)_nowr:gs << % / / 11r+ project_idc 0tj||yr5)rupdate)r;fieldss r)_setr?ks*f-r+stageexccp|d|}tjd||t|d||ty)Nz: z[pipeline] %s FAILED - %sfailed)statuserrorr@ updated_at)r rEr?r:)r;r@rAmsgs r)_errrHos5 G2cU CII):s;HCuPr+cmdlabelcwdtimeoutc  tjd|djd|Dtj|dddd||}|j dk7r*t |d |j d |jd d |S) zERun a subprocess; raise RuntimeError with a clean message on failure.z[pipeline] %s: %s c32K|]}t|ywr5)str).0cs r) z_run..xs2G3a3q63sTutf-8replace)capture_outputtextencodingerrorsrKrLrz failed (exit z): iDN)r debugr subprocessrun returncoder#stderr)rIrJrKrLresults r)_runr`usII!5#((2G32G*GH ^^ Dtgi FAeWN63D3D2ET&--X\X]J^I_`aa Mr+ audio_pathc t} tj|dddddd|gdddd }t|jj S#t $rYy wxYw) N-vrEz -show_entrieszformat=durationz-ofz"default=noprint_wrappers=1:nokey=1TrTrU)rVrWrXrYg$@)r3r[r\floatstdoutr Exception)rar-probes r)_audio_durationrhskoG dG_6G 8* FdWY  U\\'')** sAA AAz854:480z1280:720 1920:1080z 3840:2160)480p720p1080p4krcftjj|djdvS)N).jpgz.jpegz.pngz.webpz.bmprrsplitextlowerrs r) _is_imagerus+ 77  D !! $ * * ,0Z ZZr+cftjj|djdvS)Nro).mp4z.aviz.movz.mkvz.webmrqrts r) _is_videorxs+ 77  D !! $ * * ,0Y YYr+femalenaturalgenderemotionscriptlangout_pathr|r}c tjrP |} tj|||}tj |||||rt jd|||ynt j d d d lm }d }||||vr|nd } | j|t jd||y#t$r }t j d|Yd}~d}~wwxYw#t$r }t j d|Yd}~d}~wwxYw#t$r }t j d|Yd}~nd}~wwxYwtjdk(rtjdr ddddddddddd } | j|xsd j!d\} } |xsdj!dk(r| n| } |j#d d!}t%dd"| d#|d$|gd%t%t'd&d'|d(d)|gd*t(j*j-|rt)j.|t jd+| |y#t$r }t j d,|Yd}~nd}~wwxYw d dl}|j3}|j5d-d. |j7d/}|xsdj!dk(rdnd0}|D]b} t9| d1dd2t9| d3dj!}||vs|dk(rd4|vs |d0k(sAd5|vsF|j5d6| j:nn#t$rYnwxYw|j#d d7}|j=|||j?t%t'd&d'||gd8t(j*j-|rt)j.|t jd9|y:#t$r }t j d;|Yd}~nd}~wwxYwtAd}t%t'd&d?d@d'dAdBtG||g dCt j dD|yE)FzKGenerate voice audio. Returns the engine used: openai|gtts|pyttsx3|silence.)languager}z+[pipeline] speech optimisation skipped (%s)N)r|rr}z,[pipeline] TTS done via OpenAI (%s/%s) -> %sopenaiz7[pipeline] OpenAI TTS failed (%s), falling back to gTTSu[pipeline] OpenAI TTS unavailable (no key or 'openai' package not installed). Falling back to gTTS — run: pip install openai httpxr)gTTS)enhimrtateknmlgubnpauresfrdejazharptruitr)rWrz'[pipeline] TTS done via gTTS (%s) -> %sgttsz2[pipeline] gTTS failed (%s), trying macOS 'say'...darwinsay)SamanthaAlex)Lekhar)AmelieThomas)AnnaMarkus)MonicaJorge)AliceLuca)KyokoOtoya)Tingtingr)LucianaFelipe)MilenaYuri) rrrrrrrrrrrryz.mp3z _say.aiffrc-oz--z macos-say-y-i-b:a160kzsay->mp3z,[pipeline] TTS done via macOS say (%s) -> %sz3[pipeline] macOS say failed (%s), trying pyttsx3...ratevoicesmalenamerNidziradavidvoicez_tmp.wavz pyttsx3->mp3z%[pipeline] TTS done via pyttsx3 -> %spyttsx3z9[pipeline] pyttsx3 failed (%s), using silence fallback...x-flavfizanullsrc=r=44100:cl=mono-tzsilence-fallbacku[pipeline] NO VOICE: all TTS engines failed — produced SILENT audio (%ds). Install openai (pip install openai httpx) and set OPENAI_API_KEY, or ensure internet access for gTTS.silence)$ai is_enabledoptimize_for_speechrfr warningsynthesize_speechr"rrsaver0r1rrr!rsrUr`r*rrexistsremoverinit setProperty getPropertygetattrr save_to_file runAndWaitmaxminlenrP)r~rrr|r}spokener supportedtts mac_voicefemrvaiffrenginerwantmetawav_tmpdurations r) stage_ttsrs& }} VF N//wW##FHV-17DGQUW_`D Y Z MQ TY->TDI  :D(K) N I1MM N  V KKQST U U V M H!LLM ||xFLL$7 R+2D*2D)1B(0H+3E I" t|t&:&:&<>RSIC "++-9tA##FK8D %q$dF;[ I ,.$dFFHMz Zww~~d# $ HHCQ Q R KKMq Q Q RT63' ''1F & "335A8vD!!F2./q41C0DEKKM4>' " IIg  8(C T OQRSST1c#f++S12H,.$gt5O H x )*<>KK,-57 sC(B<2C(;AD< C%C C( C%%C(( D1D  D D=D88D=+CI I/I**I/3&OA.L. L.L.-O. L:7O9L::BO O>O99O>Tliveness resolutionrctj|d}|jd\}}t}t |} d} d|d|d|d} |rQt j j|r2t|r'|ddd d |d |d t| d | d dddddddddddd|g} n|rt j j|rt|r|rd} d| dzd}d| dzd }d!|d|d"|d|d#|d|d$| d%|d&|d'|d(|d)| d*}g|dd+d,d |d |d t| d-t| d.|d/d0d/d1d ddddd2ddddddd|} nR|dd+d,d |d |d t| d | d ddddddddddd|g} n-|dd3d4d d5|d(|d6| d |d t| d d7d ddddddddddd|g} t| d8tjd9| ||| S):a  Build the base MP4 from an image or video + audio. Returns duration (s). For image sources with `liveness` enabled, applies a Ken-Burns zoom plus a gentle sinusoidal pan ("head sway") so the avatar is not a frozen frame even without GPU models. ri:zscale=z*:force_original_aspect_ratio=decrease,pad=z):(ow-iw)/2:(oh-ih)/2:black,format=yuv420prz -stream_loopz-1rr-vf-c:vlibx264-presetfast-crf23-c:aaacrr -shortest -movflags +faststartzmin(1.0+0.0006*on,1.05)ziw/2-(iw/zoom/2)+sin(on/g@z )*(iw*0.006)zih/2-(ih/zoom/2)+cos(on/g@z )*(ih*0.005)z [0:v]scale=z+:force_original_aspect_ratio=increase,crop=z&,boxblur=26:3,setsar=1[bg];[0:v]scale=zU:force_original_aspect_ratio=decrease[fg];[bg][fg]overlay=(W-w)/2:(H-h)/2,zoompan=z='z':x='z':y='z':d=1:s=xz:fps=z,format=yuv420p[v]z-loop1z-rz-filter_complexz-mapz[v]z1:a20rrzcolor=c=black:s=z:r=zformat=yuv420pz make-videoz8[pipeline] Base video created (%.1fs, liveness=%s) -> %s)RES_MAPr!splitr*rhrrrrxrPrur`r r") source_pathrarrrvf_scalewhrrfpspadrIzoomryfcs r)stage_make_videorsf{{:{3H >># DAq ^Fz*H C H:AaSA CCrww~~k2y7M D.$k4 #h- Iy&&$ E66; \S[    4;9O  -D*3s7)<@A*3s7)<@AaS!%s!A3S!%"V558A3as%uM$ % %'*,02=?CEO(m%)+.s8"$&(.057=?D " $- /5 7= ?C      &  (.  0;    *  ,4 CgsD+tZc(mUC 9ffdvv{L( C D$1A!AaSC5/Q *dCM5:J Iy&&$ E66; h   lHH G x+ Or+ model_dirc|D]E}tjj||}tjj|sC|cSyr5)rrrr)rnamesnr(s r) _entrypointrFs:  GGLLA & 77>>! H r+ source_imagework_dirc `tj}|r!tjj |r|syt |gd}tjj |dd}|rtjj|stjdytjj |d} ttj|d|d|d |gd |d tjj|r|SdS#t$r }tjd |Yd}~yd}~wwxYw)z LivePortrait / SadTalker: animate a still portrait into a moving base clip. Requires a driving template video at /assets/driving.mp4. Returns animated (silent) mp4 path or None to skip. N) inference.pyzdemo.pyzrun.pyassetsz driving.mp4uL[pipeline] LivePortrait present but no entrypoint/driving asset — skippingz animated.mp4z-sz-dr liveportraitrKrLu4[pipeline] LivePortrait failed (%s) — falling back)rLIVEPORTRAIT_PATHrrisdirrrrr r"r` MODELS_PYTHONrfr)r r mdirr~drivingoutrs r)run_liveportraitrNs  D RWW]]4(\ D EFggll4=9G 0 _` '',,x 0C c|T7C(dD BggnnS)s3t3  JANs keyword args (librosa >= 0.10) • deprecated numpy aliases np.float/np.int/np.bool/np.object/np.complex -> python builtins (numpy >= 1.24) rNzaudio.pyrT)rXz=librosa\.filters\.mel\(\s*hp\.sample_rate\s*,\s*hp\.n_fft\s*,z6librosa.filters.mel(sr=hp.sample_rate, n_fft=hp.n_fft,rz<[pipeline] patched Wav2Lip audio.py mel() for modern librosaz([pipeline] could not patch audio.py (%s)z5\bnp\.(float|int|bool|object|complex)\b(?!\d|_|\s*\()rdintboolobjectcomplex)rdrrrrz.gitz.pyc,|jdS)Nro)group)mrepls r)z'_patch_wav2lip_source..sQWWQZ(8r+z&[pipeline] patched numpy aliases in %s)rerrrropenreadsubwriter r"rfrcompilewalkr)rr"audio_pysrcnewraliasroot_dfilesfr(srr s @r)_patch_wav2lip_sourcer2fsww||D*-H C 77>>( #x'2779C&&PHC czXsW5;;C@WX JJO PEUF9 6D774=b% T> A::e$ T1%A W-224II8!<6C'288;HHEqI) C >BBC(  s+BF A(F7 F4F//F47 GGface_video_or_imagectj}|rtjj |syt |ddg}t tjj |dddgxst |ddg}|r|stjdytjr t|tjj |d} ttj|d |d |d |d |d dddddddgd|dtjj|r|SdS#t$r }tjd|Yd}~yd}~wwxYw)zDWav2Lip: apply accurate lip-sync of `audio` onto `face` video/image.Nr zwav2lip_inference.py checkpointszwav2lip_gan.pthz wav2lip.pthuI[pipeline] Wav2Lip present but missing entrypoint/checkpoint — skippingz lipsync.mp4z--checkpoint_pathz--facez--audioz --outfilez --nosmoothz--resize_factorrz--pads010wav2lipru/[pipeline] Wav2Lip failed (%s) — falling back)r WAV2LIP_PATHrrrrrr r"WAV2LIP_AUTOPATCHr2r`rrrfr)r3rar rr~ckptrrs r) run_wav2lipr=s8   D RWW]]4( 0FG HF rww||D-8)=9 ; A t/? @   \] d# '',,x /C  c)># JGGH-LAr5779%%f-&AGGI2E77<<1--.+ ,  FJsC6F75AG ;AG <G #G 3G 7 GG G2G--G2in_videoc tj}|rtjj |syt |ddg}|st jdytjj|d}tj|d ttj|d|d |d d d d g d|dtj|D]P\}}}|D]E}|jjds#tjj||ccSRy#t$r } t j d| Yd} ~ yd} ~ wwxYw)z>CodeFormer / GFPGAN: per-frame face restoration & enhancement.Nzinference_codeformer.pyr uA[pipeline] CodeFormer present but missing entrypoint — skippingcf_outTexist_okrrz--bg_upsamplerNonez-wz0.7 codeformerrrrwu2[pipeline] CodeFormer failed (%s) — falling back)rCODEFORMER_PATHrrrrr r"rmakedirsr`rr(rsrrfr) rIr rr~out_dirr-r.r/r0rs r)run_codeformerrSs   D RWW]]4(  9>J KF  TUggll8X.GKK$'  cxwe56Bt % "www/OD"e779%%f-77<<a00 0  H!Ls%A'D!9#D!D!! E *EE z20:20z W-w-20:20z 20:H-h-20z W-w-20:H-h-20z(W-w)/2:(H-h)/2)ztop-leftz top-rightz bottom-left bottom-rightcenterzx=m:y=mz x=w-tw-m:y=mz x=m:y=h-th-mzx=w-tw-m:y=h-th-mzx=(w-tw)/2:y=(h-th)/2g?rTg{Gz?)opacitypositionscalerWin_pathrVrWrXrWct}tdtdt|}|xstj xsdj }|jddjddjdd} d} d } tj|td jd | } d | d | d|dtd|dzdd| } |dd|d| ddddddddddd |g}t|d!tjd"| |||y#)$z Burn a clean semi-transparent TEXT watermark (HeyGen-style), e.g. "GOLS", into every video. Font size scales with the video height; a soft shadow keeps it readable on any background. Re-encodes video, copies audio. g?g?GOLS\rr'zh*0.03zh*0.045rTrzdrawtext=text='z ':fontsize=z:fontcolor=white@z:shadowcolor=black@g333333?z.2fz:shadowx=2:shadowy=2:box=0:rrrz,format=yuv420prcopyrrrrrrrrzwatermark-textz:[pipeline] Text watermark '%s' applied (op=%.2f, %s) -> %sN) r*rrrdrWATERMARK_TEXTrrU _TEXT_POSr!r`r r")rYrrVrWrXrWropwm_textsafemarginfontsizeposdrawtextrIs r)stage_watermarkrhs6^F T3sE'N+ ,B3s))3V::d?d@|gdA|}tjdB|j dCxsdDjE}||k7r|n|xs|}t7|dEdF|dGdH/d}|dIk(rddJl#m$}t jj+|dK} |r^t?|rS|jK||| |||L\}!}"|!r| }n~tj4dM|"t7|dN|"OtM|||}nJt7|dPOtM|||}n/|dQk(rtO|||xs tM|||}n tM|||}|r|}tjdR|ntjdS|j dTdrot7|dTdUdV/tQ||}#|#rRt jj+|dW}$ tC|d8d9|#d9|d:d;dd?d@|$gdX|$}tjdYt7|ddZd[/|}%| ri tS||| |\t jjU|r%t jjW|d]kDr|}%ntj4d^d`|da}'t jj+tj,|'}(tjX|%|(d`|db})t jj+tj,|)}*t[|(|*}+t]t jjW|(dcz dd},tjde|(|,t_j`|j dfdg||j d|'|+r|)nddht]|d||||||,| ddi}-t_jb|-djdk|-djdldk|-djdm|+r dk|-djdnnddot7|dhdpdqdrdk|-djdldk|-djdm|-djt]|dtes tjdt||-dj|r7t jj|r tj|yyy#t$rYywxYw#t$r|}YZwxYw#t$r|#}YwxYw#t$r!}&tj4d_|&Yd}&~&9d}&~&wwxYw#t$rJ}.tj dutgjhtk|tm|.dvdw|.Yd}.~.d}.~.wwxYw#t$rYywxYw#|rFt jj|r& tj|w#t$rYwwxYwwwxYw)xNz[pipeline] START %szipeline] Using ffmpeg: %sz[pipeline] Project %s not found avatar_idrr~zHello, this is your AI avatar.rrrrl voice_genderryr}rzro)per_pageitemsr watermarkFwatermark_opacitywatermark_position_job_TrLz voice.mp3zbase.mp4zwatermarked.mp4 source_urlz$[pipeline] Source file not found: %s processingrzGenerating voice... )rDr@ stage_labelprogressrEr{rzNo voice was generated (silent audio). Install OpenAI (pip install openai httpx) with OPENAI_API_KEY set, or ensure internet access for gTTS, then regenerate.tts_done)r@r|voice_engine_used voice_warningvideozCreating base video...()r@r{r|r video_done:)r@r| animationz Face animation (LivePortrait)...@zanimated_av.mp4rrrr^rrrrrz mux-animatedz)[pipeline] LivePortrait animation applied lip_enginer8lipsyncz Lip sync (z)...Jdid) did_servicezdid.mp4) script_textr|rz[pipeline] D-ID failed: %szD-ID: ) did_errorz)D-ID needs an image source (not a video).rAz"[pipeline] Lip-sync applied via %suG[pipeline] No lip-sync engine produced output — using living portraitenhancez Face enhancement (CodeFormer)...Tzenhanced_av.mp4z mux-enhancedz)[pipeline] CodeFormer enhancement appliedzFinalising video...\)rVrWiu>[pipeline] watermark produced no file — using un-watermarkedu?[pipeline] watermark failed (%s) — using un-watermarked videoavatar_rwrpiz,[pipeline] Final video saved -> %s (%.2f MB)titlez Avatar Video completed)rr;rofilenamerlrD duration_secrrrprr file_size_mbrs download_urlrz /api/videos/z /downloadz/streamz /thumbnail)r stream_url thumbnail_urldoneCompleted) rDr@r{r| output_urlrvideo_idr completed_atz[pipeline] DONE %s -> video %sz[pipeline] EXCEPTION: %sr@pipeline)7r r"r*rr!rErrrrrmtreerfrr listrWATERMARK_OPACITYWATERMARK_POSITIONrPuuiduuid4r OUTPUT_DIRrQ UPLOAD_DIRbasenamerr?rrENABLE_LIVENESSrurr`rs app.servicesr generate_talkr=rHrSrhrgetsizecopy2rmroundr creater=r: traceback format_excrHr)/r;r  ffmpeg_exeprojectavatarr~rresr|r}slistsettingsuse_wm wm_opacity wm_positionjob_idrabase_mp4wm_mp4rxr candidate tts_enginerrcurrentanimatedmuxedrlip_facerrdid_outokdetailenhancedcf_av final_sourcerr final_out thumb_name thumb_path has_thumbrrrAs/ r) run_pipeliner9sHH "J/HA!^  . ; $$Z0 II7 D l x0  h'18i""7;;{B#?@FBX'GH{{:t,kk,0^,V >0JVh++i3##Q/8$58"k51\\"5s7L7LM ll#79O9OP TZZ\"2A&77<<%x0@A Ht,WW\\(K8 77<<*5h(9:ZZ -   S^^RWW5E5Ej5QRIww~~i('  BIN Z E.4 IvtZPWX   "QM ZzB*( * Zw4LWYZ#KXs-0-@-@B Z|b9 9[1 ;?" N' X>H X/@A'*dD(D* &&%%|U<=KM$GDE jj.;)BBD %17 8Ox Zy%j\6 E   0ggll8Y7Gy5(66W &v7F F%GKK >&)bggoof.E.L#)LKK `a VHD)GGLL:  \9-vhd+ WW\\#..*= #Iz: RWW__Y7)CQG  ?LY##[[.9$ [1 '0d!!(A."!+*(%  " E$K*5;-yA(t W=GP|E$K= CVZ*  Z 6z U4[M&K&uT{m7;Dkh0B&  " 1:uT{K x0  h'18  u!'&G'n!'&G' b ]_`aa bZ A -y/C/C/EF Zgz:C@@A   x0  h'  18sA b `*"L.b `9-F.b a 8(b !A(a F'b c * `65`69 ab ab ab ab  b&b<b bb cAcc/cc/ c,+c,/"d9d('d9( d41d93d44d9)NN)8__doc__rrr[r0rrrrtypingr app.configrapp.repository.storerrr r app.utils.loggerr rr rr rP__annotations__rr*r3r:r?rfrHrrCompletedProcessr`rdrhrrrurxrrrrr2r=rHrS _POSITIONSr`rhrmrr9r+r)rs0   '\\ -#(3-'c'T s  2c2.S.QSQQ9Q 6:"& d 3 Xc] 3- +5+F+F    j;k Z[C[D[ZCZDZ%YZcZZZZ/2ZCFZB;?CcCSC!$C37CCHCR3(3-3#(3-0,,,^ScSXVY]8%c%s%c%hWZm%PSCHSM:#%  ""'+   &*>#'T+S+C+"+58+ +/2+>B+FstDSDTDr+