JohnG Thanks John. With the time we will definitely make it even better. This week for example we covered classifier with hundreds of tests to understand how any instruction we change may affect the result. And because the test results were not stable we started to tune classifier. In the end we had all these tests in green (passed), stable classifier and better quality of answers.
The material we now work with is something very different. For example an incredible thing in classifier that seriously improved the stability of it out was the order of fields in JSON. Classifier should create JSON with labels, extracted identifiers and questions for RAG. I think initially it was unspecified and in the instructions labels where mentioned first, than identifiers and finally questions for the RAG. Once we specified the order with this instruction, model output became stable with improved ability to set labels and extract identifiers:
In your answer first you generate "questions", based on questions and the discussion you set "labels" and after that extract "items" identifiers.
When you work with programs you never consider how fields in JSON are ordered. You just use them. But for model it really matters. In our case it just needed some place to think of before labeling. Incredible, isn't it?
Still we are very focused on the telematics, but genAI world is absolutely exciting to explore and develop.
