A lot of good ways to split your data for RAG are dependent on the file type, for instance a Markdown-aware splitter (that can also parse out tables), or a JSON splitter. For pure unstructured text semantic chunking is a good approach. Because you can't make proper use of the… https://t.co/N3aGQZ13iW https://t.co/wGibj6mGQC
— Jerry Liu (@jerryjliu0) Mar 4, 2024
from Twitter https://twitter.com/jerryjliu0
March 04, 2024 at 04:00AM
via IFTTT
No comments:
Post a Comment