Fix issue causing file connector to fail

[Weves]

Description

https://linear.app/danswer/issue/DAN-1414/file-connector-failing-due-to-successful-doc-ids-=-setupdatable-ids

Document ID modification internal to Vespa -> returning different document IDs than passed in -> causes

        successful_doc_ids = {record.document_id for record in insertion_records}
        if successful_doc_ids != set(updatable_ids):
            raise RuntimeError(
                f"Some documents were not successfully indexed. "
                f"Updatable IDs: {updatable_ids}, "
                f"Successful IDs: {successful_doc_ids}"
            )

to fail.

Also includes some additional comments + enabling re-indexing on the file connector.

How Has This Been Tested?

Tested with a batch of files that I knew previously failed. Then verified it worked after the fix.

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

• This PR should be backported (make sure to check that the backport attempt succeeds)
• [Optional] Override Linear Check

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
internal-search	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 5, 2025 3:44am