ML/AI/LLM

Karungi

Mereko dukh hua Aapko Aisa laga, aap dm Kare mein waha aapki madad Karuna

No not sorry like we were giving an encouragement like

no need to say sorry. Just tell the person to dm

Ki I am sorry to hear this xyz aap and then tell them to dm

Jisme person needed help or was complaining

Har comment mein we were doing that

Originally toh

in these cases we can ask the person to dm us with their concern or share more in dm etc.

and share how the new bot will respond

Okay

please share what behavior ull change before making it live

Working on this

But we do need to reply to negatives hence it was not a fix

I was originally not replying to all negative because of this, llm was giving bad results

bot seems to not be handling any complaints/negative statements

Fixing

Ye llm Kaise answer kar Raha hai

https://beyond-chats.netlify.app/drpriyankamaurya.com/instagram?reply=replied

<media>

@54125194641609

this was probably the fastest i have shipped a feature, and i liked working on this thank you

https://gitlab.com/Pankaj-Baranwal/chatgpt3_backend/-/merge_requests/1480 MR for staging

this is the link to docs for how to set this up on prod

https://docs.google.com/document/d/13rtdNDT0FNnmnBFgLezbd6B12Rg0C1dd71KTnN5Bsws/edit?usp=sharing

i have shown how adding data to support tickets add data to the bug sheet too in the video

the bug sheet syncing with support tickets

Alright

<media>

these numbers are too high

<media>

<media>

@54125194641609 lets only have 2 main goals for this sprint. To reduce vector search latency by half in the first few days and to reduce answer generation latencies by half in the next.

ill do it

there were no details i thought might be a glitch

i thought this was a glitch

right?

@54125194641609 this task is still remaining

output on elefant is smaller than ivf india

but the llm replies faster as they are simple answers

due to added instructions

elefant has larger prompts

i mean on average length of final prompt of elefant vs other orgs

for uncertain

but rest are at 80k and the largest is 109k

general and pre sale are smaller about 50k chars

give me a minute

could not effectively measure this, prompt comes from the db

how much does the prompt length reduce?

14% faster about 2000 ms faster on average on the new flow without retrieval setup than ivf

how much faster?

yes elefant's prompt classification speeds up the process even more

https://app.notion.com/p/Elefant-Classifier-39ea4e78beb980b7b9ecfaec66e9d508?source=copy_link

elefant test for chats if elefant's prompt is faster than other org move towards elefant's prompt architecture agreed upon optimization: Language detection create the link between support tickets and the bug sheet

- UI issue when adding a support ticket as regression case: critical bug vs realtime check yes/no; 500 error in /draft - dev/staging -> if new chat is clicked on embed then it really should create a new chat - analyze trace of elefant vs other orgs. If elefant's send_message is faster than other orgs then we move other orgs to its architecture

in waiting room

http://meet.google.com/cec-nozs-dew

Okay

make one

One

Meet link? Or should I make on

@111561876209776

Let’s connect 4:30, I just woke up

or 4:30

can we connect at 3?

@111561876209776

On dr Priyanka regex rules are setup to not reply if the user’s latest message is an Instagram reel, but that is only for Instagram DMs

And not use any AI tokens

Ideally, it should not respond at all

@54125194641609 / @111561876209776 How will our chatbot respond to these msgs?

can we connect either today or tomorrow to discuss these results? @111561876209776

test flow result

https://docs.google.com/spreadsheets/d/1drnTrCfGj_7ME1oyi_NFSBSADnB3hWfui5Evcxbic_E/edit?usp=sharing

original flow results

https://docs.google.com/spreadsheets/d/1ZWdt60cgBljB5tynAfatdaiZnKe2GRqMNJJgTKRYDHQ/edit?usp=sharing

this has my research report for the video bot and beyond chats node comparison

https://app.notion.com/p/Research-Results-39da4e78beb980608b42dc2fbf40942d?source=copy_link

we should only keep the user's info such as name and email etc, rest should be a blank slate

fixed

we should not send chat history from previous chat if user is starting a new chat

this is a new chat

after conversation is restarted bot is still fed the data from the previous conversation as conversation history

<media>

@111561876209776

one conversation is not enough for testing. I need you to gather questions/conversations of startup founders from reddit AMAs/other forums and then test the bot on it

Checking

I was doing it from a tier 2/3 city founder pov

@54125194641609 what was the user persona that you used for testing?

should have been hyperlinked

<media>

you can build a shareable link to the chatbot this way: https://integration.beyondchats.com/demo/start-up-mentors-6a54dd503602d/startupmentors.in

Okay

that is not needed for god users

i have added task details

Lemme check if they are even visible on my product view or not

@54125194641609 fill task updates column: - ⁠⁠The best medical care chatbot should be able to respond any medical question. Dont keep it restricted to RAG. - ⁠We need a chatbot for startupmentors.in Trained on their website data.

“node_prompt” org prompt are very vague

+ a small tooltip to tell what this section of the prompt is supposed to do and how it will affect the overall prompt

Spent a lot of time on this for medical appraisals, best medical care and SUM

Macros such as {{sales intent}} we should be able to edit them too per org, or atleast rather than just showing macro in the editor we should show what that macro would be adding to that section

On the same container I feel color coded sections would be much easier to distinguish between from

On the right side container where the entire prompt is visible, clicking on a section should either directly open that section’s editor or let me edit there only

Me too

i find it very overwhelming

@54125194641609 any suggestions for UI/UX of prompt engineering page? since you have used it alot now

Yes this would be very helpful

okay. will be adding lead scoring prompt to the dashboard as well in the next sprint

And is that the repo where we directly talk to the database without any pre or post processing?

@102095734710467 can you add me as a collaborator on beyondchats-answer-engine ?

But classifying that this behaviour should be high/low/medium gives better results than a prompt like if user is asking about price but not contact or if user is only asking about contact

Depends org to org

is there a fixed structure of the rubric?

I did this during dr Priyanka and here

I feel like making a clear rubric (non contradictory) improves the lead scoring much more than it is

Oh

no just showing the input. Not your task, am just sharing my thoughts

??

Like reasoning for low high or medium

oh okay. we need better visibility on dashboard for what exactly was used to generate the LLM outcomes

@111561876209776

.

high

this specific chat gave higg

{ "personal_info": { "phone": null, "email": "Khejihad@yahoo.com", "name": "Kheedeer Jiihaad", "location": "Mauritius", "age": null }, "main_topics": [ "IVF Treatment" ], "pain_points": "Payment for the consultation failed initially. The user is not receiving email updates regarding the treatment progress. User's wife is experiencing gas and bloating post-procedure.", "lead_score": "HIGH", "service_interested_in": [ "IVF", "Consultations", "Egg/Embryo Freezing" ] }

HIGH — at, or clearly committed to, the consultation end state: - A consultation / Zoom / video call or appointment is booked, scheduled, or already attended. - Consultation fee paid, OR explicit commitment to book one ("want to consult with Dr Malpani", "schedule my consultation", "I want to proceed/book"). - Concrete consultation-readiness: a named treatment (IVF/ICSI/egg or embryo freezing/PGD) PLUS a clear next step toward consulting (timeline, travel-to-clinic plan, or explicit request to be contacted to set it up). MEDIUM — a genuine new prospect, not yet committed to a cons…

scoring rubric:

these are of the recent 50 chats i extracted from using our god dev tools

ivf india lead scoring on the dynamic lead scoring prompt

https://docs.google.com/spreadsheets/d/1dRPaa_rznt_hq7vUkwwCgGHfLnacV5v99mMQdA5aZUo/edit?usp=sharing

what is your email? i will add you to org team

@111561876209776

can you test i have setup the org

I’ll set it up through the dashboard then

Did it last time through merge I forgot I had god options

u could have setup the org via dashboard itself

forgot to add my email 🥲

merge it there and send the link to access that

php artisan org:provision-startupmentors --dry-run # preview first php artisan org:provision-startupmentors

created a merge on the prod

Creating the prompts

@54125194641609 hows the progress?

moving over to setting up the org

i went through the content myself too and fixed some wrong transcriptions

this is the csv data with the transripted content

https://docs.google.com/spreadsheets/d/1sGRKUIGPm1WWhm2p1BD_E8M2avpU_E7Degz6R5hpeDc/edit?usp=sharing

alright

yes i saw

only the resources ones, they are under 2 mins

alright

corrected this in task details

wait, the resources page is https://startupmentors.in/resources/ and not https://startupmentors.in/podcasts

emon ive added exact URLs to pages

it will have huge transcripts

these yt videos are like 1 hr long each

transcripts on videos?

have this setup on prod directly

okay

@54125194641609

please see to this task on priority:

yes its yours only. Its in the task updates

i do not have access to this

this is my my csv?

Alright

expecting the sheet to answer how the new prompt is fixing all lead scoring related bugs for ivfindia

For Della and elefant on priority

Yes we wanted custom scoring form

also, i dont have enough confidence in making custom lead scoring for ivfindia live. Can you please share the analysis sheet? https://docs.google.com/spreadsheets/d/1qrlpr4DsB6H4xSH0Qji0W0NEHJnTTubQ21PCLIw-a38/edit?gid=967869485#gid=967869485 this one only has runs for della resorts and elefant

Alright

@54125194641609 hey can you run this via the new lead scoring prompt u had developed for ivfindia and tell me the output

How is this low intent?

https://beyond-chats.netlify.app/ivfindia.com/leads?page=1&show_ai_analysis=111395

updated the bot's behaviour for this

alright

The counter-question is not at all relevant. Will not help improve user engagement. Chatbot needs to ask better questions please - ex: try to understand user’s situation better / is user facing any issues that led them to come here, etc

updated the guardrail

But it used guardrail 2, interesting

<media>

It starts from however

I feel iss response mein bot ne aadha reply Diya hai

Aagya

can you refresh

This org disappeared from my dashboard

these guardrails still exist on prod

are the changes not live on prod?

@54125194641609 the best medical care chatbot still cannot answer all medical questions:

All fixed only flirt phased as praise slips

<media>

<media>

on it

@54125194641609

@111561876209776 are these fixed for Dr. Priyanks?

Sent inv on GitHub

Okay

this is medical appraisals

Which org is this?

updated the link

Don’t have access to the file in task details

<media>

Okay

@54125194641609 I will delegated an urgent bug fix with ETA as today. please check