Process completed with exit code 1.

Process completed with exit code 1.

The self-hosted runner: tt-metal-ci-vm-21 lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.

Process completed with exit code 2.

Unable to download artifact(s): Artifact not found for name: eager-dist-ubuntu-20.04-any-profiler Please ensure that your artifact is not expired and the artifact was uploaded using a compatible version of toolkit/upload-artifact. For more information, visit the GitHub Artifacts FAQ: https://github.com/actions/toolkit/blob/main/packages/artifact/docs/faq.md

Process completed with exit code 1.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

Disk usage is high. Will try pruning stuff.

Expected None, but tests/nightly/single_card/functional_unet/experimental/functional_unet/tests/test_unet_trace.py::test_unet_trace_2cq_same_io_multi_device[wormhole_b0-True-1-2-128-device_params0-True] returned UNetPerformanceStatistics(groups=2, batch=1, num_devices=2, inference_and_compile_time=0.3347928524017334, inference_time=0.0030369162559509277), which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`?

Expected None, but tests/nightly/single_card/functional_unet/experimental/functional_unet/tests/test_unet_trace.py::test_unet_trace_2cq_same_io_multi_device[wormhole_b0-True-1-2-128-device_params0-False] returned UNetPerformanceStatistics(groups=2, batch=1, num_devices=2, inference_and_compile_time=0.4310035705566406, inference_time=0.0024107880890369415), which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`?

Expected None, but tests/nightly/single_card/functional_unet/experimental/functional_unet/tests/test_unet_trace.py::test_unet_trace_2cq_same_io[1-2-128-device_params0] returned UNetPerformanceStatistics(groups=2, batch=1, num_devices=1, inference_and_compile_time=0.26326918601989746, inference_time=0.002387598156929016), which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`?

Expected None, but tests/nightly/single_card/functional_unet/experimental/functional_unet/tests/test_unet_trace.py::test_unet_trace_2cq_same_io_multi_device[wormhole_b0-True-1-2-128-device_params0-True] returned UNetPerformanceStatistics(groups=2, batch=1, num_devices=2, inference_and_compile_time=0.3347928524017334, inference_time=0.0030369162559509277), which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`?

Expected None, but tests/nightly/single_card/functional_unet/experimental/functional_unet/tests/test_unet_trace.py::test_unet_trace_2cq_same_io_multi_device[wormhole_b0-True-1-2-128-device_params0-False] returned UNetPerformanceStatistics(groups=2, batch=1, num_devices=2, inference_and_compile_time=0.4310035705566406, inference_time=0.0024107880890369415), which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`?

Expected None, but tests/nightly/single_card/functional_unet/experimental/functional_unet/tests/test_unet_trace.py::test_unet_trace_2cq_same_io[1-2-128-device_params0] returned UNetPerformanceStatistics(groups=2, batch=1, num_devices=1, inference_and_compile_time=0.26326918601989746, inference_time=0.002387598156929016), which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`?

The hookimpl pytest_addhooks uses old-style configuration options (marks or attributes). Please use the pytest.hookimpl(tryfirst=True) decorator instead to configure the hooks. See https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers

The hookimpl pytest_addhooks uses old-style configuration options (marks or attributes). Please use the pytest.hookimpl(tryfirst=True) decorator instead to configure the hooks. See https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

The use of `x.T` on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error in a future release. Consider `x.mT` to transpose batches of matrices or `x.permute(*torch.arange(x.ndim - 1, -1, -1))` to reverse the dimensions of a tensor. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3637.)

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

The use of `x.T` on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error in a future release. Consider `x.mT` to transpose batches of matrices or `x.permute(*torch.arange(x.ndim - 1, -1, -1))` to reverse the dimensions of a tensor. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3637.)

No files were found with the provided path: generated/test_reports/. No artifacts will be uploaded.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

Expected None, but tests/nightly/single_card/functional_unet/experimental/functional_unet/tests/test_unet_trace.py::test_unet_trace_2cq_same_io[1-2-128-device_params0] returned UNetPerformanceStatistics(groups=2, batch=1, num_devices=1, inference_and_compile_time=0.5917031764984131, inference_time=0.002377880737185478), which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`?

Expected None, but tests/nightly/single_card/functional_unet/experimental/functional_unet/tests/test_unet_trace.py::test_unet_trace_2cq_same_io[1-2-128-device_params0] returned UNetPerformanceStatistics(groups=2, batch=1, num_devices=1, inference_and_compile_time=0.5917031764984131, inference_time=0.002377880737185478), which will be an error in a future version of pytest. Did you mean to use `assert` instead of `return`?

The hookimpl pytest_addhooks uses old-style configuration options (marks or attributes). Please use the pytest.hookimpl(tryfirst=True) decorator instead to configure the hooks. See https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers

The hookimpl pytest_addhooks uses old-style configuration options (marks or attributes). Please use the pytest.hookimpl(tryfirst=True) decorator instead to configure the hooks. See https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers

Hugepages service not found. Using old rc.local method

Hugepages service not found. Using old rc.local method

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

No files were found with the provided path: /work/generated/test_reports/. No artifacts will be uploaded.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

The use of `x.T` on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error in a future release. Consider `x.mT` to transpose batches of matrices or `x.permute(*torch.arange(x.ndim - 1, -1, -1))` to reverse the dimensions of a tensor. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3637.)

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

Hugepages service not found. Using old rc.local method

To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).

To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

Expected None, but models/demos/wormhole/mamba/demo/demo.py::test_demo[user_input0-state-spaces/mamba-2.8b-slimpj-50-device_params0] returned DemoResult(generated_text=['The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was first settled by the Iroquois, who were followed by the French, and then the English, who were followed by the Americans, who were followed by the Canadians, who were followed by the Indians, who were followed by', 'The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was first settled by the Iroquois, who were followed by the French, and then the English, who were followed by the Americans, who were followed by the Canadians, who were followed by the Indians, who were followed by', 'The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was first settled by the Iroquois, who were followed by the French, and then the English, who were followed by the Americans, who were followed by the Canadians, who were followed by the Indians, who were followed by', 'The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was first settled by the Iroquois, who were followed by the French, and then the English, who were followed by the Americans, who were followed by the Canadians, who were followed by the Indians, who were followed by', 'The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was fi

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

Expected None, but models/demos/wormhole/mamba/demo/demo.py::test_demo[user_input0-state-spaces/mamba-2.8b-slimpj-50-device_params0] returned DemoResult(generated_text=['The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was first settled by the Iroquois, who were followed by the French, and then the English, who were followed by the Americans, who were followed by the Canadians, who were followed by the Indians, who were followed by', 'The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was first settled by the Iroquois, who were followed by the French, and then the English, who were followed by the Americans, who were followed by the Canadians, who were followed by the Indians, who were followed by', 'The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was first settled by the Iroquois, who were followed by the French, and then the English, who were followed by the Americans, who were followed by the Canadians, who were followed by the Indians, who were followed by', 'The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was first settled by the Iroquois, who were followed by the French, and then the English, who were followed by the Americans, who were followed by the Canadians, who were followed by the Indians, who were followed by', 'The city of Sarnia is located on the eastern shore of Lake Huron at its extreme southern point where it flows into the St. Clair River . Most of the surrounding area is flat , and the elevation ranges from 169 metres ( 554 ft ) and 281 metres ( 922 ft ) above sea level . The soil mostly comprises clay . Despite this high percentage of clay , the soil is remarkably rich for cultivation . Prior to the Ice Age , glaciers covered most of the area , as can be seen not only by the existence of the Great Lakes themselves but also of alluvial sand deposits, terminal moraines, and rich oil reserves.\nThe area was fi

Support for class-based `config` is deprecated, use ConfigDict instead.

Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details

Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details

Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details

Hugepages service not found. Using old rc.local method

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

`top_k_top_p_filtering` is scheduled for deletion in v4.39. Use `TopKLogitsWarper` and `TopPLogitsWarper` instead.

Disk usage is 71 %

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 52 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 57 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 76 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 57 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 53 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 79 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 82 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 51 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 77 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 80 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Disk usage is 80 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 80 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 65 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 55 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 56 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 70 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 67 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 61 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Disk usage is 59 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 78 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Disk usage is 75 %

Touching and printing out SMI info

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 68 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 77 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 75 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 80 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Disk usage is 81 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 77 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Hugepages is now setup.

Disk usage is 59 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages is now setup.

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available

Disk usage is 46 %

Touching and printing out SMI info

tt-smi reset was successful

Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available