INTERNAL Webmaster

Hmmm ok. Maybe later

Let's use gemini or some other video tool for this , instead of actually shooting a video , we can try it out , it might work

Maybe we can try something that might fix this wobbly head issue with drmalpani avatar only, we can generate a video with drmalpani photo which satisfies all the requirements that are needed for their avatar processing and generate one more avatar.

And retraining k liye kya chahiye hoga?

Okayy

Wobbly head km tha usme

Let's switch back to previous avatar then

Updates - - Green - on , off , when speaking - three dots ( Completed and Deployed on live prod ) - Ringing sound during call connecting ( Completed and Deployed ) - Head wobbling too much ( Actually we can't do anything for this, the only way we can solve it is by creating a new avatar for drmalpani and testing that - so basically trial and error, nothing we can do code wise ). - Moving on to the landing page creation

Let me know when I can test this again please

What's your expectation?

Waise sir, u said about increment? Is this still on ? 🥲 . I have become a bit poor .

<media>

<media>

Jitna to normal video call me nhi le leta hai

And it takes about 4-5 sec only

But we need this slowness cause connection establish hone se phle thoda backend me prewarm functions chalte hai so that jb start ho to koi bhi delay naa ho

very slow

<media>

Test it out one more time and lemee know if anthing else needs improvement

I have been making difffernt account to get the credits for the avatar since the credits in the beyondchats liveavatar account have been consumed, so please test the agent accordingly , cause for the free ones that i have used for now , it only gives very less credits wchih gets consumed pretty fast.

Well for this kind of this , i know an approach which can reduce it a bit , but it requires testing and optimization and will take time. Basically we can use something like a noise filter for it , but i have no idea how to implement it , i have just seen it in a code somewhere before

Test it out once again . Changes done - - Wired elevenlabs TTS with an indian male voice for the chatbot. - Added a modal for asking permissions for both microphone and camera before the session so the user can join in with the permissions enabled. - Fixed some bugs and optimization for latency since i have swapped from opeai realtime to elevenlabs as the tts. SO it is now a 3 stage pipeline of STT, LLM and TTS along with the VAD

https://dr-malpani-nurse.vercel.app/

Maine audio files 32-33 minutes ka upload kr diya hia on elevenlabs , now drmlpani has to complete the consent step and starts the cloning process .

I'll start work from second half of the day, as I have some work at my house .

Yep , should work, just the duration we have to manage to make atleast 30 min or more for Professional voice clone.

These shorter versions will work?

<media>

<media>

https://1drv.ms/u/c/1e94d7fde4d41a06/IQBvwOhdY-AFSq1AUpw1jBVcASbdboaElPJZzfNQRlhN3ro?e=g6P1bh

But didn't receive for tbmc

Good

<media>

No specific section showing raw recorded videos

This is what is there , nothing else

Not ai generated avatar videos

I'm saying raw malpani recorded videos

Arre

Anyways we can use it .

There's no raw recording stored anywhere on heygen , whatever training data he may have used that are for training avatar and voice

Okay I have these avatar videos jinhe I can download and the extract their audio to use , all are 2 min or smaller .

Lemme check

His recording are there on heygen

You're saying his recordings can't be reused?

He has recorded himself a lot of audio and video for training heygen

I meant that he has multiple voices of his on heygen sure , but we can't use that in our pipeline, having normal audio would more better for using eleven labs for cloning.

He has trained his heygen avatar for both video clone as well as voice clone

Yes

I have messaged devanshu sir for the audio files anyways

I can see his avatar, but these are cloned voices , using cloned voice to generate clone ?

Can you check his heygen account?

This is Dr Malpani

Which I generated for training it

Lots of audio content on HeyGen

Ok cool. Once you’re done with that, ping me and I will ask Dr Malpani to complete the consent

Phle audio files upload kr skte hai

Voice clone start hone se phle last step is consent

Ya pehle consent?

Han, but that will come later na?

Okay , then you would have to have drmalpani sir, do the consent thing

Please ask Devanshu to use Dr Malpani’s youtube channel to get you 30 minutes of DrMalpani’s voice recording

Nope , waise if you just go to the create voice option, all the steps are listed. So basically you need to either upload your audio files in which you can speak anything. And once wo ho jaye then eleven labs needs consent jisme wo live script record krke voice match krta hai before cloning the voice to preven unautho use

And is there a specific script that needs to be recorded? Or any audio of that person will do?

Or live recording?

30 minutes of recorded audio is fine?

Two options for voice clone- One is Instant clone that requires only 10 seconds of audio jo ki direct record kr ke and then consent match krke create kr skte hai . and second is Professional voice clone which requiires at least 30 minutes of audio and then consent to create .

Great

Yes done

Code changes every 15 sec

Done?

530705

970401

Code

2FA

``` contactbeyondchats@gmail.com tF^qKE>DLT6t7%S ``` ElevenLabs creds

Voice clone ka next step kya hai?

I had to completely redo the whole thing with a python based livekit agent worker service at the backend so that the delay is as minimum as possible along witht the the typescript frontend. So usme time lag gya . But i think architecture , we can stick on this. Ispe jo bhi add krna hoga we can add it gradually.

1. For the female voice - I have just used for now for testing , we can change it. Not an issue. 2. Using the same prompt that you told to use from before, so we maybe need to have some more constraints in the proompt for telling it when to switch to hindi. 3. Okay, I will have an notification at the start to ask for permission , instead of leaving it to the browser to ask for permissions for mic and video at the start. 4. For the ambient noise, we can fix it but it is more so of an optimization problem that we can tune as we go 5. FOr the avatar of drmalpani , we can use it immeditaley since…

When can we start using Dr Malpani's avatar and voice?

It also catches ambient noise as words sometimes. Any way to fix this?

Please ask for permissions in the start of the session instead of asking for video when I turn on video and asking for microphone when I turn on microphone

Why is voice female while avatar is male?

Why is cost in hindi?

<media>

New link for testing the video chatbot - https://dr-malpani-nurse.vercel.app/

Achaa , will fix that

Is tool k meta tags bhi sahi nhi hain.

https://drmalpani.com/embryo-app/ Have you already checked this before?

Ok

Pehle video chatbot, uske baad embryo analysis - depending on your bandwidth.

And the frontend displays all 3 predictions.

Also, Embyo Image Analysis project hai - within DrMalpani. Just check once kaise kaam kr rha hai. When we pass an image for analysis, abhi sirf ek Deep Learning model hai, usse analysis ho rhi hain redults a rhe. But what I want is ki same image hum log Gemini & ChatGPT ko bhi bhejein. And vo bhi analyse kr k apne responses dein.

Sharing in 2 min

Yes , sir elevenlabs ke creds please share kr dijiye

@117111057871026 Video chatbot issues fix kr lena. I had informed on Saturday. And phase 2 as well.

Done

Can you ask claude through tg to do this?

Okay , so abhi korean time ke according Krna hai right. I'll do it

I want ki unke timezone k according update ho jaye article updation + digest

So Dr. Malpani is in Korea right now. And will be in SF in 3 days

Hmm to whi problem tha hmara timezone asia/kolkata set hai for thr app and just wo timing 22:30 ho gaya tha , 10:30 IST isliye kal wo notification gaya tha with only one published article and since article update raat mein 2 baje aaspaas hota hai so us time pe np new articles were updated

yes

Aapne timing change Kiya tha email and WhatsApp notification ka through telegram bot .

Please re-auth

Recharge done

Subscription end ho gya tha

Your organization has disabled Claude subscription access for Claude Code · Use an Anthropic API key instead, or ask your admin to enable access

No idea about it as i didn't change anything. The production claude is not working

Then why was a new empty mail sent?

These are the articles updated today jinka notification aaya tha subah

But article update happenes at 2 -3 am, and notification for that is sent at 7:15 as usual , this is new scheduled articles maybe added recently cause last wale finish ho gaye the jime publish time subah ke jagah raat ka kiya hua hai

<media>

0 articles updated today?

U have to tag the bot using @ to send it any message.

How do I know if this worked or not?

At least the text and audio should start to nudge the user and let them know the conversation has started and they need to start talking

The video bot should start talking immediately and not just smile dumbly

I muted myself, phir bhi it was able to listen to what i was sayinh!!!

Did you not check this

I can hear the audio while lip sync is delayed

Prompt for thebestmedicalcare pathways generation - You generate medical decision flowcharts that look like roadmap.sh/frontend in STYLE only — soft pastel topic boxes on a vertical centre spine, cream subtopics branching left and right via curved dotted lines, occasional white description cards, edge labels on conditional decisions ("Yes"/"No"/"If fever"). The CONTENT is real clinical decision-making for patients. TASK: Build a comprehensive, patient-friendly clinical decision flowchart for "${ailment}" as roadmap.sh-style JSON. The chart must be detailed enough that someone with no medica…

This is just for phase 1 , some things can be made better when we refine/optimze it further later on.

I haven't got the prompt that you guys were using , i dmed both yash and sid, didn't get any reply

Chatbot for testing .

https://video-chatbot-taupe.vercel.app/

Minutes - - Video calling chatbot - Phase 1 : - Can be used on mobile and desktop screen seamless - Similar to google meet captions for transcription on mobile screens - Google meet centric UI Phase 2 : - Multilingual support (Indian accent English and Hindi with natural tone ) Phase 3 : - Integrate with BeyondChats vector DB to fetch data , tool callings etc. - Modify the prompt for care pathways generation in thebestmedicalcare.com for Indian version of the pathways for which the TA includes the tier 2 and tier3 users/patients. - Move out the telegram bot into the machin…

Ok

call kr lena

ok

Can we do it 5 mins later ?

quick call?

avatar creation ke liye de diya hai

Sir , Malpani Sir ka voice clone ke liye , we need to have him do it only . Process aise hai ki - you have to upload the audio files , then click to consent and then the elevenlabs records the person voice live at that time and verifies that it is the same person whose is speaking in the audio files. Once that's done , then it clones it.

Today's Tasks - - Video calling chatbot - Having custom avatar and professional voice clone of drmalpani on eleven labs that can be used for video calling chatbot. ( High priority ) - Move on to remaining tasks once the chatbot thing is done. ( QA automation and the task to have chatbot fetch data from all the micro tools or even tool to talk internally with one another )

Not on personal chat

@117111057871026 Let’s have all conversations here from now on.