Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(search): Adds logic to download search results #4893

Merged
merged 38 commits into from
Mar 11, 2025
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
90c8dc9
feat(search): Adds ratelimiter to throttle csv exports per day
ERosendo Jan 6, 2025
ca4f4f5
feat(settings): Adds setting to limit number of exported search results
ERosendo Jan 15, 2025
8cda135
feat(search): Adds email template for search results export
ERosendo Jan 15, 2025
93d31a2
refactor(search): Moves helper functions to search_utils.py
ERosendo Jan 15, 2025
d4b2f1a
feat(search): Adds task to send email with search results as attachment
ERosendo Jan 15, 2025
92cddf5
feat(search): Adds view to export search results
ERosendo Jan 16, 2025
0f2c992
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Jan 17, 2025
f8f5fee
Merge branch 'main' into 599-feat-add-logic-to-download-results
albertisfu Jan 20, 2025
b4cf486
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Jan 21, 2025
fe3e21f
feat(search): Adds docket search type to CSV generation helpers
ERosendo Jan 21, 2025
a66cdb4
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Jan 23, 2025
c9925a0
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Feb 20, 2025
9249217
feat(search): Introduce `get_csv_headers` for CSV export
ERosendo Feb 20, 2025
5f469ca
feat(lib): Adds camelCase to snake_case conversion helper
ERosendo Feb 20, 2025
58d4a98
feat(search): Convert search result keys to snake_case
ERosendo Feb 20, 2025
42067f7
feat(lib): Refines parent-child result combination logic
ERosendo Feb 24, 2025
407374b
feat(search): Adds query link in CSV results email
ERosendo Feb 24, 2025
727bb94
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Feb 24, 2025
1016003
feat(lib): Includes empty fields in CSV export fetching logic.
ERosendo Feb 25, 2025
a4ac467
feat(lib): Introduces CSVSerializableDocumentMixin to simplify export
ERosendo Feb 27, 2025
f12c240
feat(search): Implement CSV header and transformation methods
ERosendo Feb 27, 2025
423f3c8
refactor(lib): Include data transformations in search result header r…
ERosendo Feb 27, 2025
c31dcf3
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Feb 28, 2025
f573ce9
fix(search): Adds missing classmethod decorator to docket doc helper
ERosendo Feb 28, 2025
0cdbfc2
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Mar 5, 2025
b25788f
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Mar 5, 2025
375450a
feat(lib): Adds new argument to the do_es_search helper
ERosendo Mar 5, 2025
cf4958e
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Mar 5, 2025
0339135
Merge branch 'main' into 599-feat-add-logic-to-download-results
albertisfu Mar 6, 2025
523c0a9
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Mar 10, 2025
0c292cc
feat(search): Adds default values for CSV header and data transformat…
ERosendo Mar 10, 2025
945a538
feat(search): Adds document type in CSV export.
ERosendo Mar 10, 2025
b6e5350
feat(search): Adds missing fields in judge search export
ERosendo Mar 11, 2025
1143e37
feat(search): Improves data transformation to handle missing fields
ERosendo Mar 11, 2025
503453d
feat(search): Add test for dockets without documents
ERosendo Mar 11, 2025
ef59d25
Merge branch 'main' into 599-feat-add-logic-to-download-results
ERosendo Mar 11, 2025
bd2c574
feat(csv-export): Refactor retry mechanism for CSV export task.
ERosendo Mar 11, 2025
2b80a3e
Merge branch 'main' into 599-feat-add-logic-to-download-results
albertisfu Mar 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions cl/lib/elasticsearch_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,40 @@ def wrapper_func(*args, **kwargs) -> Any:
return wrapper_func


class CSVSerializableDocumentMixin:

@classmethod
def get_csv_headers(cls) -> list[str]:
"""
Returns a list of strings representing the headers for a CSV file.

This method defines the column headers for a CSV representation of the
data associated with this class.

:return: A list of strings, where each string is a column header.
"""
raise NotImplementedError(
"Subclass must implement get_csv_headers method"
)

@classmethod
def get_csv_transformations(cls) -> dict[str, Callable[..., Any]]:
"""
Generates a dictionary of transformation functions for CSV export.

This method defines how specific fields in a data structure should be
transformed before being written to a CSV file. It covers
transformations for various fields, including those from list of fields
with highlights, file paths, URLs, and renamed fields.

:return: A dictionary where keys are field names and values are lambda
functions that define the transformations.
"""
raise NotImplementedError(
"Subclass must implement get_csv_transformations method"
)


def build_numeric_range_query(
field: str,
lower_bound: int | float,
Expand Down
6 changes: 6 additions & 0 deletions cl/lib/ratelimiter.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ def get_path_to_make_key(group: str, request: HttpRequest) -> str:
if "test" in sys.argv:
ratelimiter_all_2_per_m = lambda func: func
ratelimiter_unsafe_3_per_m = lambda func: func
ratelimiter_unsafe_5_per_d = lambda func: func
ratelimiter_unsafe_10_per_m = lambda func: func
ratelimiter_all_10_per_h = lambda func: func
ratelimiter_unsafe_2000_per_h = lambda func: func
Expand All @@ -79,6 +80,11 @@ def get_path_to_make_key(group: str, request: HttpRequest) -> str:
rate="3/m",
method=UNSAFE,
)
ratelimiter_unsafe_5_per_d = ratelimit(
key=get_ip_for_ratelimiter,
rate="5/d",
method=UNSAFE,
)
ratelimiter_unsafe_10_per_m = ratelimit(
key=get_ip_for_ratelimiter,
rate="10/m",
Expand Down
Loading