— 05/08/2024 at 03:29:14 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 22
Updated At: 2024-08-05T15:40:04.089Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 05/08/2024 at 03:40:04 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 23
Updated At: 2024-08-06T15:48:47.118Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation. To install and use this method please see: [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md "User Notes").
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 06/08/2024 at 03:48:47 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-08-06T15:48:46.930Z
Content Updated:
id: cell-key-perturbation-r
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet— 06/08/2024 at 03:48:46 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 4
Updated At: 2024-08-06T16:20:39.327Z
Content Updated:
id: cell-key-perturbation-r
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet— 06/08/2024 at 04:20:39 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 24
Updated At: 2024-08-07T08:43:43.313Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method please see: [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md "User Notes").
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 07/08/2024 at 08:43:43 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 5
Updated At: 2024-08-07T08:43:43.375Z
Content Updated:
id: cell-key-perturbation-r
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet— 07/08/2024 at 08:43:43 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 25
Updated At: 2024-08-07T09:23:16.673Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method please see: [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md "User Notes").
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
additionalInformation:
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 07/08/2024 at 09:23:16 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 6
Updated At: 2024-08-07T09:23:16.711Z
Content Updated:
id: cell-key-perturbation-r
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet— 07/08/2024 at 09:23:16 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 26
Updated At: 2024-08-08T10:52:36.438Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method please see: [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md "User Notes").
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
additionalInformation:
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 08/08/2024 at 10:52:36 AM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 7
Updated At: 2024-08-08T12:09:57.628Z
Content Updated:
id: cell-key-perturbation-r test
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet— 08/08/2024 at 12:09:57 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 27
Updated At: 2024-08-08T12:09:57.593Z
Content Updated:
id: cell-key-perturbation test
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method please see: [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md "User Notes").
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
additionalInformation:
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 08/08/2024 at 12:09:57 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 8
Updated At: 2024-08-08T12:19:23.620Z
Content Updated:
id: cell-key-perturbation-r
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet— 08/08/2024 at 12:19:23 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 28
Updated At: 2024-08-08T12:19:23.582Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method please see: [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md "User Notes").
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
additionalInformation:
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 08/08/2024 at 12:19:23 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 29
Updated At: 2024-08-08T12:25:36.531Z
Content Updated:
id: cell-key-perturbation test
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method please see: [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md "User Notes").
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
additionalInformation:
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 08/08/2024 at 12:25:36 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 9
Updated At: 2024-08-08T12:25:36.598Z
Content Updated:
id: cell-key-perturbation-r test
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet— 08/08/2024 at 12:25:36 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 10
Updated At: 2024-08-08T12:35:15.775Z
Content Updated:
id: cell-key-perturbation-r
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet— 08/08/2024 at 12:35:15 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 30
Updated At: 2024-08-08T12:35:15.572Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method please see: [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md "User Notes").
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
additionalInformation:
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 08/08/2024 at 12:35:15 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 1
Updated At: 2024-08-19T13:28:09.376Z
Content Updated:
id: cell-key-perturbation-r
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: R
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
additionalInformation: — 19/08/2024 at 01:28:09 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-08-20T08:21:01.380Z
Content Updated:
id: ratio-of-means
name: Ratio of Means
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Ratio of means is a standard imputation method used for business surveys. The method imputes for each non-responding contributor a single numeric target variable within the dataset for multiple periods simultaneously. It uses the relationship between the target variable of interest and a predictive value and/or auxiliary variable to inform the imputed value. Due to its robust nature, it does not use any form of trimming or outliering.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/ratio_of_means/methodological_specification.md— 20/08/2024 at 08:21:01 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 19
Updated At: 2024-08-20T08:21:01.332Z
Content Updated:
id: mean-of-ratios
name: Mean of Ratios
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md— 20/08/2024 at 08:21:01 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 21
Updated At: 2024-08-20T11:08:12.796Z
Content Updated:
id: mean-of-ratios
name: Mean of Ratios
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md— 20/08/2024 at 11:08:12 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 1
Updated At: 2024-08-20T11:13:14.531Z
Content Updated:
id: mean-of-ratios
name: Mean of Ratios
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md— 20/08/2024 at 11:13:14 AM —
Editor: Jason Bell
Environment: preprod
Content Type: heroHomePage
Revision: 31
Updated At: 2024-08-21T13:08:49.789Z
Content Updated:
heroTitle: An open source library for statistical code approved by the ONS (rollback testing)
heroButton: Find methods— 21/08/2024 at 01:08:49 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 1
Updated At: 2024-08-21T13:52:10.075Z
Content Updated:
id: mean-of-ratios
name: Mean of Ratios
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md— 21/08/2024 at 01:52:10 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-08-21T14:59:09.376Z
Content Updated:
id: mean-of-ratios
name: Mean of Ratios
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md— 21/08/2024 at 02:59:09 PM —
Editor: null null
Environment: preprod
Change Type: DeletedEntry
Entry ID: 1HJAAO0pl0PBHmxrTumJaI
This entry was deleted
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-08-22T14:02:03.888Z
Content Updated:
id: mean-of-ratios
name: Mean of Ratios
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md— 22/08/2024 at 02:02:03 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 4
Updated At: 2024-08-22T15:44:04.899Z
Content Updated:
id: mean-of-ratios
name: Mean of Ratios
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md— 22/08/2024 at 03:44:04 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 5
Updated At: 2024-08-23T14:07:42.128Z
Content Updated:
id: mean-of-ratios
name: Mean of Ratios
theme: Imputation
expertGroup: Editing & Imputation
language: Python/PySpark
author: ONS
description: Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.
As imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Released Yet
additionalInformation: Links to the method code and specification are provided for further reading.
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md— 23/08/2024 at 02:07:42 PM —
Editor: Jason Bell
Environment: preprod
Content Type: heroHomePage
Revision: 32
Updated At: 2024-08-27T08:39:48.791Z
Content Updated:
heroTitle: An open source library for statistical code approved by the ONS
heroButton: Find methods— 27/08/2024 at 08:39:48 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 15
Updated At: 2024-08-27T09:02:29.049Z
Content Updated: entry 1HJAAO0pl0PBHmxrTumJaI has been deleted.
id: {"en-US":"mean-of-ratios"}
name: {"en-US":"Mean of Ratios"}
theme: {"en-US":"Imputation"}
expertGroup: {"en-US":"Editing & Imputation"}
language: {"en-US":"Python/PySpark"}
author: {"en-US":"ONS"}
description: {"en-US":"Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.\n\nAs imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period."}
contactDetails: {"en-US":"smlhelp@ons.gov.uk"}
methodReady: {"en-US":false}
releaseVersion: {"en-US":"Not Released Yet"}
additionalInformation: {"en-US":"Links to the method code and specification are provided for further reading."}
specificationLink: {"en-US":"https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md"}— 27/08/2024 at 09:02:29 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-27T09:25:35.798Z
Content Updated:
id: package
term: Package
meaning: An archive containing software and additional metadata that is distributed by a package manager. Sometimes used as a synonym for 'library', but does not always refer to libraries.— 27/08/2024 at 09:25:35 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-08-27T09:25:28.581Z
Content Updated:
id: coding-standards
title: Coding standards
content: Code will be written to internationally recognised standards and according to principles set out in these links:-
* [Government Digital Service](https://gds-way.cloudapps.digital/#the-gds-way/)
* [ONS Quality Assurance for Code and Analysis (Duck book)](https://best-practice-and-impact.github.io/qa-of-code-guidance/intro.html)
* [ONS Digital Software Engineering Standards](https://github.com/ONSdigital/software-engineer-community/blob/master/standards/README.md)
For Python, this means it will adhere to the [PEP8 Python style guide](https://peps.python.org/pep-0008/) and the [GDS Python style guide.](https://gds-way.cloudapps.digital/manuals/programming-languages/python/python.html#content)— 27/08/2024 at 09:25:28 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-08-27T09:25:26.512Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.1.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.1.0
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/thousand_pounds_correction/thousand_pounds_correction.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md— 27/08/2024 at 09:25:26 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-08-27T09:25:28.581Z
Content Updated:
id: coding-standards
title: Coding standards
content: Code will be written to internationally recognised standards and according to principles set out in these links:-
* [Government Digital Service](https://gds-way.cloudapps.digital/#the-gds-way/)
* [ONS Quality Assurance for Code and Analysis (Duck book)](https://best-practice-and-impact.github.io/qa-of-code-guidance/intro.html)
* [ONS Digital Software Engineering Standards](https://github.com/ONSdigital/software-engineer-community/blob/master/standards/README.md)
For Python, this means it will adhere to the [PEP8 Python style guide](https://peps.python.org/pep-0008/) and the [GDS Python style guide.](https://gds-way.cloudapps.digital/manuals/programming-languages/python/python.html#content)— 27/08/2024 at 09:25:28 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-08-27T09:25:29.069Z
Content Updated:
id: ht-ratio-estimation
name: Horvitz-Thompson Ratio Estimator
theme: Sample Design & Estimation
expertGroup: Sample Design & Estimation
language: Python/PySpark
author: ONS
description: Sampling in business surveys is often done using stratified simple random sampling without replacement. The largest businesses are usually selected with certainty. This SML method uses two approaches for estimating totals in such cases. The Horvitz-Thompson estimator in this case is known as a stratified expansion estimator, which is the simplest method used. We also use a combined ratio estimator. This is appropriate where there is a linear relationship through the origin between the target variable and an auxiliary variable we know for all units, and the scatter about the line increases with the size of the auxiliary variable.
contactDetails: smlhelp@ons.gov.uk
methodReady: false
releaseVersion: Not Ready Yet
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/sample_design_and_estimation/estimation/ht_estimation.rst— 27/08/2024 at 09:25:29 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-27T09:25:32.925Z
Content Updated:
id: github
term: GitHub
meaning: A popular Microsoft-owned service providing hosting for Git repositories, a Web interface to Git, and related services.
relatedlist: ["Git"]— 27/08/2024 at 09:25:32 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 2
Updated At: 2024-08-27T09:29:13.030Z
Content Updated:
id: version_control
term: Version Control
meaning: A means of tracking changes made to a file or collection of files. Commonly used to track the development of software projects. TESTING INTEGRATION— 27/08/2024 at 09:29:13 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 3
Updated At: 2024-08-27T09:37:41.051Z
Content Updated:
id: version_control
term: Version Control
meaning: A means of tracking changes made to a file or collection of files. Commonly used to track the development of software projects.— 27/08/2024 at 09:37:41 AM —
Editor: Jason Bell
Environment: preprod
Content Type: helpCentreInformation
Revision: 1
Updated At: 2024-08-27T11:50:54.277Z
Content Updated:
id: view-methods
title: Find and view methods
content: You can find and view methods within the [methods catalogue](/methods)
Where available, links to GitHub for source code and specifications will be on the method's details page.— 27/08/2024 at 11:50:54 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 2
Updated At: 2024-08-27T11:51:06.786Z
Content Updated: entry 7IvVbD5tUwbzWeWPqwQP5n has been deleted.
id: {"en-US":"semantic_versioning"}
term: {"en-US":"Semantic Versioning"}
meaning: {"en-US":"A popular scheme for determining version numbers. Consists of three integers, separated by full stops. The first integer is the 'major' version number, the second is the 'minor' version number, and the third is the 'patch' number. The patch number is incremented by 1 when bugfixes are implemented. The minor version is incremented by 1 when new functionality is introduced without affecting compatibility with previous versions. The major version number is incremented by 1 when changes are made that break compatibility with previous versions. See https://semver.org"}
externalLinkTextShort: {"en-US":"Semantic Versioning"}
externalUrl: {"en-US":"https://semver.org"}— 27/08/2024 at 11:51:06 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-27T11:51:02.800Z
Content Updated:
id: git
term: Git
meaning: A popular software tool for version control. Files are stored in repositories (or 'repos'), which can be local or remote, and changes made are committed to a log, allowing files to be reverted to previous versions. A project can have multiple contributing authors and multiple 'branches' (variants) simultaneously, with Git providing tools to merge changes.— 27/08/2024 at 11:51:02 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-27T11:51:04.847Z
Content Updated:
id: metadata
term: Metadata
meaning: 'Data about data', giving information about a product itself, as opposed to its contents. In the case of the SML, metadata includes, for example: when a method was onboarded; its development status; by whom it was developed; whether or not it is approved for use in official outputs; etc.— 27/08/2024 at 11:51:04 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-27T11:51:01.027Z
Content Updated:
id: expert_group
term: Expert Group
meaning: A group of ONS employees within the Methodology and Quality Directorate with expertise in a specific area. For a full list, see https://analysisfunction.civilservice.gov.uk/government-statistical-service-and-statistician-group/gss-support/methodology/information-on-specific-methods/
externalLinkTextShort: Expert group
externalUrl: https://analysisfunction.civilservice.gov.uk/government-statistical-service-and-statistician-group/gss-support/methodology/information-on-specific-methods/— 27/08/2024 at 11:51:01 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-27T11:51:01.551Z
Content Updated:
id: community_methods
term: Community Methods
meaning: Code developed by a wider community of analysts. These methods are held in a community code repository to ensure there is a clear distinction in terms of maintenance of the code. After sign-off, any changes to methods held in the SML community repository must be made via the SML governance process to ensure that specifications and documentation are kept up to date and users are informed of changes to the code. The developers of the code would be accountable for making any updates to the code.— 27/08/2024 at 11:51:01 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-27T11:51:08.832Z
Content Updated:
id: third_party_methods
term: Third Party Methods
meaning: Third party methods are defined as those where a package, piece of code or software is developed and maintained outside of the SML strictures. This may include packages developed within ONS but where code is held and managed in a separate repository, or packages developed and maintained externally.— 27/08/2024 at 11:51:08 AM —
Editor: Jason Bell
Environment: preprod
Content Type: about
Revision: 1
Updated At: 2024-08-27T12:28:04.971Z
Content Updated:
about: # About the Statistical Methods Library
## What is the Statistical Methods Library?
The statistical methods library is a central library for approved statistical methods for use within ONS and wider.
## Overall mission
The Statistical Methods Library (SML) provides a central resource for consistent and reproducible statistical methods within the ONS and wider. The library ensures that methods for official statistical outputs are fully transparent. It provides proportionate governance and assurance processes, including peer review of methods for statistical production and analysis or exploration.
The library facilitates continuous improvement by providing users with the ability to add new methods or request changes to existing ones. It contains documentation on method specifications, how-to guidance and contact details for help and support. The SML will allow users to share methods and best practice with other government departments.
## Current objectives
- To act as a shared single point of reference for best practice methods and share methods across government departments.
- To continuously update methods based on peer reviews, emerging facts and user feedback.
- To provide clear evidence of audit and quality assurance for approved methods.
## Future objectives
- To standardise best practice methods across the ONS and government departments.
- To act as a developmental tool, providing reference material and guides to approved methods.
- To demonstrate, share and promote innovation in methodological practice across GSS/wider government.
## Further information
The SML contains many features, including:
- A catalogue of methods
- Statistical methods specifications
- A code repository
- Downloadable Methods packages
- Technical methods governance
- A glossary of terms
- How-to guidance— 27/08/2024 at 12:28:04 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-28T08:48:33.485Z
Content Updated:
id: pypi
term: PyPI
meaning: The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software.
relatedlist: ["SML","Package","Repository"]
externalLinkTextShort: ONS-SML user profile on PyPI
externalUrl: https://pypi.org/user/ons-sml/— 28/08/2024 at 08:48:33 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 23
Updated At: 2024-08-28T13:31:54.564Z
Content Updated: entry 6WK1Ynt7VFw7lGHiOpJVcg has been deleted.
id: {"en-US":"pypi"}
term: {"en-US":"PyPI"}
meaning: {"en-US":"The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software."}
relatedlist: {"en-US":["SML","Package","Repository"]}
externalLinkTextShort: {"en-US":"ONS-SML user profile on PyPI"}
externalUrl: {"en-US":"https://pypi.org/user/ons-sml/"}— 28/08/2024 at 01:31:54 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-08-28T14:05:45.688Z
Content Updated:
id: pypi
term: PyPI
meaning: The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software.
relatedlist: ["SML","Package","Repository"]
externalLinkTextShort: ONS-SML user profile on PyPI
externalUrl: https://pypi.org/user/ons-sml/— 28/08/2024 at 02:05:45 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-08-28T14:39:01.048Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/PySpark
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 28/08/2024 at 02:39:01 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 17
Updated At: 2024-08-30T16:37:31.690Z
Content Updated: entry 1PQFWq5r92jnyKqZnqtmrk has been deleted.
id: {"en-US":"cell-key-perturbation"}
name: {"en-US":"Cell Key Perturbation"}
theme: {"en-US":"Statistical Disclosure Control"}
expertGroup: {"en-US":"Statistical Disclosure Control"}
language: {"en-US":"Python/PySpark"}
author: {"en-US":"ONS"}
description: {"en-US":"Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.\n\nTo install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md)."}
contactDetails: {"en-US":"smlhelp@ons.gov.uk"}
methodReady: {"en-US":true}
releaseVersion: {"en-US":"v2.0.0"}
releaseLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0"}
codeLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py"}
userDocumentationLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md"}— 30/08/2024 at 04:37:31 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-08-30T16:39:18.965Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/PySpark
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 30/08/2024 at 04:39:18 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-09-03T14:53:17.093Z
Content Updated: entry viM9TL5rdzMZITGCQBIbe has been deleted.
id: {"en-US":"mean-of-ratios"}
name: {"en-US":"Mean of Ratios"}
theme: {"en-US":"Imputation"}
expertGroup: {"en-US":"Editing & Imputation"}
language: {"en-US":"Python/PySpark"}
author: {"en-US":"ONS"}
description: {"en-US":"Mean of Ratios imputation is a standard imputation method for business surveys. It can be used to impute value for unit (complete) non-response or item (partial) non-response. There is an option to use trimming as the method can be influenced by extreme values. The method imputes a single numeric variable. It uses the relationship between the variable being imputed and an appropriate predictive or auxiliary variable.\n\nAs imputation can be carried out for multiple periods simultaneously, the method can apply forward, backward or construction imputation. The type of imputation used will vary for each non-respondent in each period depending on whether data is available in the predictive period."}
contactDetails: {"en-US":"smlhelp@ons.gov.uk"}
methodReady: {"en-US":false}
releaseVersion: {"en-US":"Not Released Yet"}
specificationLink: {"en-US":"https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/imputation/mean_of_ratios/technical_specification.md"}— 03/09/2024 at 02:53:17 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-09-03T14:53:17.093Z
Content Updated: entry 68ETsCxLbjjKpd0j9fuvWk has been deleted.
id: {"en-US":"ht-ratio-estimation"}
name: {"en-US":"Horvitz-Thompson Ratio Estimator"}
theme: {"en-US":"Sample Design & Estimation"}
expertGroup: {"en-US":"Sample Design & Estimation"}
language: {"en-US":"Python/PySpark"}
author: {"en-US":"ONS"}
description: {"en-US":"Sampling in business surveys is often done using stratified simple random sampling without replacement. The largest businesses are usually selected with certainty. This SML method uses two approaches for estimating totals in such cases. The Horvitz-Thompson estimator in this case is known as a stratified expansion estimator, which is the simplest method used. We also use a combined ratio estimator. This is appropriate where there is a linear relationship through the origin between the target variable and an auxiliary variable we know for all units, and the scatter about the line increases with the size of the auxiliary variable."}
contactDetails: {"en-US":"smlhelp@ons.gov.uk"}
methodReady: {"en-US":false}
releaseVersion: {"en-US":"Not Ready Yet"}
specificationLink: {"en-US":"https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/sample_design_and_estimation/estimation/ht_estimation.rst"}— 03/09/2024 at 02:53:17 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 4
Updated At: 2024-09-03T15:36:14.128Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 03/09/2024 at 03:36:14 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 22
Updated At: 2024-09-03T15:50:12.134Z
Content Updated: entry 1PQFWq5r92jnyKqZnqtmrk has been deleted.
id: {"en-US":"cell-key-perturbation"}
name: {"en-US":"Cell Key Perturbation"}
theme: {"en-US":"Statistical Disclosure Control"}
expertGroup: {"en-US":"Statistical Disclosure Control"}
language: {"en-US":"Python/Pandas"}
author: {"en-US":"ONS"}
description: {"en-US":"Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.\n\nTo install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md)."}
contactDetails: {"en-US":"smlhelp@ons.gov.uk"}
methodReady: {"en-US":true}
releaseVersion: {"en-US":"v2.0.0"}
releaseLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0"}
codeLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py"}
userDocumentationLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md"}— 03/09/2024 at 03:50:12 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 5
Updated At: 2024-09-03T15:50:12.134Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 03/09/2024 at 03:50:12 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 23
Updated At: 2024-09-03T16:00:06.739Z
Content Updated: entry 1PQFWq5r92jnyKqZnqtmrk has been deleted.
id: {"en-US":"cell-key-perturbation"}
name: {"en-US":"Cell Key Perturbation"}
theme: {"en-US":"Statistical Disclosure Control"}
expertGroup: {"en-US":"Statistical Disclosure Control"}
language: {"en-US":"Python/Pandas"}
author: {"en-US":"ONS"}
description: {"en-US":"Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.\n\nTo install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md)."}
contactDetails: {"en-US":"smlhelp@ons.gov.uk"}
methodReady: {"en-US":true}
releaseVersion: {"en-US":"v2.0.0"}
releaseLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0"}
codeLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py"}
userDocumentationLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md"}— 03/09/2024 at 04:00:06 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 6
Updated At: 2024-09-03T16:20:20.824Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 03/09/2024 at 04:20:20 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 28
Updated At: 2024-09-03T16:36:03.269Z
Content Updated: entry 1PQFWq5r92jnyKqZnqtmrk has been deleted.
id: {"en-US":"cell-key-perturbation"}
name: {"en-US":"Cell Key Perturbation"}
theme: {"en-US":"Statistical Disclosure Control"}
expertGroup: {"en-US":"Statistical Disclosure Control"}
language: {"en-US":"Python"}
author: {"en-US":"ONS"}
description: {"en-US":"Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.\n\nTo install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md)."}
contactDetails: {"en-US":"smlhelp@ons.gov.uk"}
methodReady: {"en-US":true}
releaseVersion: {"en-US":"v2.0.0"}
releaseLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0"}
codeLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py"}
userDocumentationLink: {"en-US":"https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md"}— 03/09/2024 at 04:36:03 PM —
Editor: Elinor Everitt
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 7
Updated At: 2024-09-03T16:55:18.182Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python/Pandas
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 03/09/2024 at 04:55:18 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 2
Updated At: 2024-09-04T13:04:11.576Z
Content Updated:
id: pypi
term: PyPI
meaning: The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software. Test
relatedlist: ["SML","Package","Repository"]
externalLinkTextShort: ONS-SML user profile on PyPI
externalUrl: https://pypi.org/user/ons-sml/— 04/09/2024 at 01:04:11 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 4
Updated At: 2024-09-04T13:05:22.151Z
Content Updated:
id: pypi
term: PyPI
meaning: The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software. test
relatedlist: ["SML","Package","Repository"]
externalLinkTextShort: ONS-SML user profile on PyPI test
externalUrl: https://pypi.org/user/ons-sml/— 04/09/2024 at 01:05:22 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 5
Updated At: 2024-09-04T13:05:47.796Z
Content Updated:
id: pypi
term: PyPI
meaning: The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software.
relatedlist: ["SML","Package","Repository"]
externalLinkTextShort: ONS-SML user profile on PyPI
externalUrl: https://pypi.org/user/ons-sml/— 04/09/2024 at 01:05:47 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: glossaryEntry
Revision: 5
Updated At: 2024-09-04T13:05:47.796Z
Content Updated:
id: pypi
term: PyPI
meaning: The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software.
relatedlist: ["SML","Package","Repository"]
externalLinkTextShort: ONS-SML user profile on PyPI
externalUrl: https://pypi.org/user/ons-sml/— 04/09/2024 at 01:05:47 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 2
Updated At: 2024-09-16T08:59:56.513Z
Content Updated:
id: install-a-method
title: Install a method
content: # Install a method
### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter `pip install sml_small`.
3. To install a specific version enter `pip install sml_small==<version_number>`. For example, `pip install sml_small==1.1.0`. Search "sml_small" on “PyPI · The Python Package Index” for available versions.
4. To use Date Adjustment or Selective Editing run the following: `import sml_small.date_adjustment as date_adjust` or `import sml_small.selective_editing as seled`.
5. To use Totals and Components or Thousand Pounds run the following: `from sml_small.editing import thousand_pounds` or `from sml_small.editing import totals_and_components`.
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation)
2. From your Python environment enter `pip install <method_name>`. For example, the Python version of cell-key-perturbation is installed using `pip install cell_key_perturbation`.
3. To install a specific version enter `pip install <method_name>==<version_number>`. For example, `pip install cell_key_perturbation==1.1.1`. Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
1) Install the method in your project using `renv::install("<method_name>")`. For example, if the method was named "aggregate" you would enter `renv::install("aggregate")`.
2) To install a specific version use `renv::install("package@version")`. For example, to install "aggregate 1.2.1" you would use `renv::install("aggregate@1.2.1")`.
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
1) Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
2) At the prompt enter `conda create <environment_name>`. You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
3) Enter `conda activate <environment_name>`.
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment. To leave the environment at any time enter `conda deactivate`. To come back to your environment later, simply repeat step 3.
5) Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
4) Now, enter `conda install python=<version_number>`. If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in this environment after successfully installing Python. If not, enter `conda install pip`.
5) Install required packages using `pip install <package_name>==<version>`. For example, `pip install pandas==1.5.3`. If you want to check what is currently installed in your environment enter `conda list`.
If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering “conda deactivate” from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
1) Enter `pip install spyder-kernels` (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels. To do this enter `pip install spyder-kernels=1.9.2`.
2) Open Spyder.
3) From the top menu, go to `Tools, Preferences, Python Interpreter`.
4) Select `Use the following Python Interpreter`.
5) Click on the `select file` icon (to the right of the address bar below).
6) Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter `conda list` from within your virtual environment. The file path you need will appear at the top of the list.
7) Click on the folder to open your virtual environment and scroll down to find and click on the python.exe file.
8) Click `Apply` and `OK` to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
9) When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
1) Enter `pip install ipykernel` (still within Anaconda prompt and your virtual environment).
2) Then enter `python -m ipykernel install --name <env_name>`.
3) Start Jupyter Notebook.
4) From the `Files` page (you should be here by default) go to `New`, located in the top right.
5) Click on `New` and select your virtual environment name.
6) Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
1) Open R Studio and select `File -> New Project`.
2) Select `New Directory -> New Project` and give your directory a useful name.
3) Tick `Use renv with this project`.
4) Consult the User Documentation for details about what packages are required to run a method. Make note of these.
5) Install required package versions using `renv::install("package@version")`. For example, to install `devtools 2.4.4` use `renv::install("devtools@2.4.4")`.
6) Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using `renv` will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using `sessionInfo()`. — 16/09/2024 at 08:59:56 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreStructure
Revision: 2
Updated At: 2024-09-17T10:04:27.120Z
Content Updated:
structure: {"categories":[{"name":"information","label":"Information","subcategories":[{"name":"methods-request","label":"Submit a method request"},{"name":"version-methods","label":"How the methods are versioned"},{"name":"coding-standards","label":"Coding standards"}]},{"name":"access","label":"Access (and usage)","subcategories":[{"name":"view-methods","label":"Find and view methods"},{"name":"install-a-method","label":"Install a method"}]},{"name":"feedback","label":"Feedback","subcategories":[{"name":"report-bug","label":"Report a defect or bug"},{"name":"provide-feedback","label":"Provide feedback"}]},{"name":"support","label":"Support","subcategories":[{"name":"support","label":"Get support"},{"name":"expert-groups","label":"Get information on expert groups"},{"name":"troubleshooting","label":"Troubleshooting"},{"name":"github","label":"Using GitHub"}]}]}— 17/09/2024 at 10:04:27 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-09-17T12:35:16.998Z
Content Updated:
id: install-a-method
title: Install a method
content: # Install a method
### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter ```pip install sml_small```.
3. To install a specific version enter ```pip install sml_small==version_number```. For example, ```pip install sml_small==1.1.0```. Search "sml_small" on “PyPI · The Python Package Index” for available versions.
4. To use Date Adjustment or Selective Editing run the following: ```import sml_small.date_adjustment as date_adjust``` or ```import sml_small.selective_editing as seled```.
5. To use Totals and Components or Thousand Pounds run the following: ```from sml_small.editing import thousand_pounds``` or ```from sml_small.editing import totals_and_components```.
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation)
2. From your Python environment enter ```pip install method_name```. For example, the Python version of cell-key-perturbation is installed using ```pip install cell_key_perturbation```.
3. To install a specific version enter ```pip install method_name==version_number```. For example, ```pip install cell_key_perturbation==1.1.1```. Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
1) Install the method in your project using ```renv::install("method_name")```. For example, if the method was named "aggregate" you would enter ```renv::install("aggregate")```.
2) To install a specific version use ```renv::install("package@version")```. For example, to install "aggregate 1.2.1" you would use ```renv::install("aggregate@1.2.1")```.
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
1) Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
2) At the prompt enter ```conda create environment_name```. You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
3) Enter ```conda activate environment_name```.
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment. To leave the environment at any time enter ```conda deactivate```. To come back to your environment later, simply repeat step 3.
5) Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
4) Now, enter ```conda install python=version_number```. If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in this environment after successfully installing Python. If not, enter ```conda install pip```.
5) Install required packages using `pip install <package_name>==<version>`. For example, ```pip install pandas==1.5.3```. If you want to check what is currently installed in your environment enter ```conda list```.
If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering “conda deactivate” from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
1) Enter ```pip install spyder-kernels``` (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels. To do this enter ```pip install spyder-kernels=1.9.2```.
2) Open Spyder.
3) From the top menu, go to ```Tools, Preferences, Python Interpreter```.
4) Select ```Use the following Python Interpreter```.
5) Click on the ```select file``` icon (to the right of the address bar below).
6) Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter ```conda list``` from within your virtual environment. The file path you need will appear at the top of the list.
7) Click on the folder to open your virtual environment and scroll down to find and click on the python.exe file.
8) Click ```Apply``` and ```OK``` to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
9) When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
1) Enter ```pip install ipykernel``` (still within Anaconda prompt and your virtual environment).
2) Then enter ```python -m ipykernel install --name env_name```.
3) Start Jupyter Notebook.
4) From the ```Files``` page (you should be here by default) go to ```New```, located in the top right.
5) Click on `New` and select your virtual environment name.
6) Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
1) Open R Studio and select ```File -> New Project```.
2) Select ```New Directory -> New Project``` and give your directory a useful name.
3) Tick ```Use renv with this project```.
4) Consult the User Documentation for details about what packages are required to run a method. Make note of these.
5) Install required package versions using ```renv::install("package@version")```. For example, to install ```devtools 2.4.4``` use ```renv::install("devtools@2.4.4")```.
6) Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using ```renv``` will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using `sessionInfo()`. — 17/09/2024 at 12:35:16 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 4
Updated At: 2024-09-18T13:25:15.646Z
Content Updated:
id: install-a-method
title: Install a method
content: # Install a method
### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter <code> pip install sml_small </code>.
3. To install a specific version enter <code> pip install sml_small==version_number </code>. For example, <code> pip install sml_small==1.1.0 </code>. Search "sml_small" on “PyPI · The Python Package Index” for available versions.
4. To use Date Adjustment or Selective Editing run the following: <code> import sml_small.date_adjustment as date_adjust </code> or <code> import sml_small.selective_editing as seled </code>.
5. To use Totals and Components or Thousand Pounds run the following: <code> from sml_small.editing import thousand_pounds</code> or <code> from sml_small.editing import totals_and_components </code>.
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation)
2. From your Python environment enter <code> pip install method_name </code>. For example, the Python version of cell-key-perturbation is installed using <code> pip install cell_key_perturbation```.
3. To install a specific version enter <code> pip install method_name==version_number </code>. For example, <code> pip install cell_key_perturbation==1.1.1 </code>. Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
1) Install the method in your project using <code> renv::install("method_name") </code>. For example, if the method was named "aggregate" you would enter <code> renv::install("aggregate")</code>.
2) To install a specific version use <code> renv::install("package@version")</code>. For example, to install "aggregate 1.2.1" you would use <code> renv::install("aggregate@1.2.1")</code>.
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
1) Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
2) At the prompt enter <code> conda create environment_name </code>. You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
3) Enter <code> conda activate environment_name </code>.
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment. To leave the environment at any time enter <code> conda deactivate </code>. To come back to your environment later, simply repeat step 3.
5) Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
4) Now, enter <code> conda install python=version_number </code>. If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in this environment after successfully installing Python. If not, enter <code> conda install pip </code>.
5) Install required packages using <code> pip install package_name==version </code>. For example, <code> pip install pandas==1.5.3 </code>. If you want to check what is currently installed in your environment enter <code> conda list </code>.
If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering <code> conda deactivate </code> from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
1) Enter <code> pip install spyder-kernels </code> (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels. To do this enter <code> pip install spyder-kernels=1.9.2 </code>.
2) Open Spyder.
3) From the top menu, go to <code> Tools, Preferences, Python Interpreter </code>.
4) Select <code> Use the following Python Interpreter </code>.
5) Click on the <code> select file </code> icon (to the right of the address bar below).
6) Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter <code> conda list </code> from within your virtual environment. The file path you need will appear at the top of the list.
7) Click on the folder to open your virtual environment and scroll down to find and click on the python.exe file.
8) Click <code> Apply </code> and <code> OK </code> to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
9) When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
1) Enter <code> pip install ipykernel </code> (still within Anaconda prompt and your virtual environment).
2) Then enter <code> python -m ipykernel install --name env_name </code>.
3) Start Jupyter Notebook.
4) From the <code> Files </code> page (you should be here by default) go to ```New```, located in the top right.
5) Click on <code> New </code> and select your virtual environment name.
6) Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
1) Open R Studio and select <code> File -> New Project </code>.
2) Select <code> New Directory -> New Project </code> and give your directory a useful name.
3) Tick <code> Use renv with this project </code>.
4) Consult the User Documentation for details about what packages are required to run a method. Make note of these.
5) Install required package versions using <code> renv::install("package@version") </code>. For example, to install devtools 2.4.4 use <code> renv::install("devtools@2.4.4") </code>.
6) Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using <code> sessionInfo() </code>. — 18/09/2024 at 01:25:15 PM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 7
Updated At: 2024-09-25T10:05:49.135Z
Content Updated:
id: sml
term: SML
meaning: The Statistical Methods Library, a centralised collection of methods approved for use by the ONS. Comprises a code library, accessed through GitHub, and a collection of associated documentation, accessed via the SML Portal.— 25/09/2024 at 10:05:49 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 7
Updated At: 2024-09-25T10:05:49.135Z
Content Updated:
id: sml
term: SML
meaning: The Statistical Methods Library, a centralised collection of methods approved for use by the ONS. Comprises a code library, accessed through GitHub, and a collection of associated documentation, accessed via the SML Portal.— 25/09/2024 at 10:05:49 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 10
Updated At: 2024-09-26T13:11:17.942Z
Content Updated:
id: smll
term: SML
meaning: The Statistical Methods Library, a centralised collection of methods approved for use by the ONS. Comprises a code library, accessed through GitHub, and a collection of associated documentation, accessed via the SML Portal.— 26/09/2024 at 01:11:17 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 5
Updated At: 2024-09-30T11:06:14.457Z
Content Updated:
id: install-a-method
title: Install a method
content: # Install a method
### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**IDS Users:**
IDS users should follow the IDS guidance for downloading and installing SML methods.
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter: __pip install sml_small__
3. To install a specific version enter: __pip install sml_small==version_number__
For example: __pip install sml_small==1.1.0__
Search "sml_small" on “PyPI · The Python Package Index” for available versions.
4. To use Date Adjustment or Selective Editing run the following: __import sml_small.date_adjustment as date_adjust__ OR __import sml_small.selective_editing as seled__
5. To use Totals and Components or Thousand Pounds run the following: __from sml_small.editing import thousand_pounds__ OR __from sml_small.editing import totals_and_components__
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter __pip install method_name__
For example, the Python version of cell-key-perturbation is installed using __pip install cell_key_perturbation__
3. To install a specific version enter __pip install method_name==version_number__
For example, __pip install cell_key_perturbation==1.1.1__
Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
1) Install the method in your project using __renv::install("method_name")__
For example, if the method was named "aggregate" you would enter __renv::install("aggregate")__
2) To install a specific version use __renv::install("package@version")__
For example: __renv::install("aggregate@1.2.1")__
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
1) Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
2) At the prompt enter __conda create environment_name__
For example, to create and environment called "test_env" you would enter __conda create test_env__
You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
3) Enter __conda activate environment_name__
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment.
To leave the environment at any time enter __conda deactivate__ To come back to your environment later, simply repeat step 3.
5) Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
4) Now, enter __conda install python=version_number__ If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in your environment after successfully installing Python. If not, enter __conda install pip__
5) Install required packages using __pip install package_name==version__.
For example, __pip install pandas==1.5.3__
If you want to check what is currently installed in your environment enter __conda list__
6) If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering __conda deactivate__ from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
1) Enter __pip install spyder-kernels__ (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels.
To do this enter __pip install spyder-kernels=1.9.2__
2) Open Spyder.
3) From the top menu, go to __Tools, Preferences, Python Interpreter__.
4) Select __Use the following Python Interpreter__.
5) Click on the __select file__ icon (to the right of the address bar below).
6) Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter __conda list__ from within your virtual environment. The file path you need will appear at the top of the list.
7) Click on the folder to open your virtual environment and scroll down to find and click on the __python.exe file__.
8) Click __Apply__ and __OK__ to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
9) When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
1) Enter __pip install ipykernel__ (still within Anaconda prompt and your virtual environment).
2) Then enter __python -m ipykernel install --name env_name__
3) Start Jupyter Notebook.
4) From the __Files__ page (you should be here by default) go to ```New```, located in the top right.
5) Click on __New__ and select your virtual environment name.
6) Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
1) Open R Studio and select __File -> New Project__.
2) Select __New Directory -> New Project__ and give your directory a useful name.
3) Tick __Use renv with this project__.
4) Consult the User Documentation for details about what packages are required to run a method. Make note of these.
5) Install required package versions using __renv::install("package@version")__
For example, to install devtools 2.4.4 use __renv::install("devtools@2.4.4")__
6) Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using __sessionInfo()__ — 30/09/2024 at 11:06:14 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 6
Updated At: 2024-09-30T11:31:14.313Z
Content Updated:
id: install-a-method
title: Install a method
content: # Install a method
### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**IDS Users:**
IDS users should follow the IDS guidance for downloading and installing SML methods.
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter: __pip install sml_small__
3. To install a specific version enter: __pip install sml_small==version_number__
For example: __pip install sml_small==1.1.0__
Search "sml_small" on “PyPI · The Python Package Index” for available versions.
4. To use Date Adjustment or Selective Editing run the following: __import sml_small.date_adjustment as date_adjust__ OR __import sml_small.selective_editing as seled__
5. To use Totals and Components or Thousand Pounds run the following: __from sml_small.editing import thousand_pounds__ OR __from sml_small.editing import totals_and_components__
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter __pip install method_name__
For example, the Python version of cell-key-perturbation is installed using __pip install cell_key_perturbation__
3. To install a specific version enter __pip install method_name==version_number__
For example, __pip install cell_key_perturbation==1.1.1__
Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
1) Install the method in your project using __renv::install("method_name")__
For example, if the method was named "aggregate" you would enter __renv::install("aggregate")__
2) To install a specific version use __renv::install("package@version")__
For example: __renv::install("aggregate@1.2.1")__
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
1) Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
2) At the prompt enter __conda create environment_name__
For example, to create and environment called "test_env" you would enter __conda create test_env__
You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
3) Enter __conda activate environment_name__
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment.
To leave the environment at any time enter __conda deactivate__ To come back to your environment later, simply repeat step 3.
5) Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
4) Now, enter __conda install python=version_number__ If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in your environment after successfully installing Python. If not, enter __conda install pip__
5) Install required packages using __pip install package_name==version__.
For example, __pip install pandas==1.5.3__
If you want to check what is currently installed in your environment enter __conda list__
6) If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering __conda deactivate__ from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
1) Enter __pip install spyder-kernels__ (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels.
To do this enter __pip install spyder-kernels=1.9.2__
2) Open Spyder.
3) From the top menu, go to __Tools, Preferences, Python Interpreter__.
4) Select __Use the following Python Interpreter__.
5) Click on the __select file__ icon (to the right of the address bar below).
6) Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter __conda list__ from within your virtual environment. The file path you need will appear at the top of the list.
7) Click on the folder to open your virtual environment and scroll down to find and click on the __python.exe file__.
8) Click __Apply__ and __OK__ to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
9) When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
1) Enter __pip install ipykernel__ (still within Anaconda prompt and your virtual environment).
2) Then enter __python -m ipykernel install --name env_name__
3) Start Jupyter Notebook.
4) From the __Files__ page (you should be here by default) go to ```New```, located in the top right.
5) Click on __New__ and select your virtual environment name.
6) Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
1) Open R Studio and select __File -> New Project__.
2) Select __New Directory -> New Project__ and give your directory a useful name.
3) Tick __Use renv with this project__.
4) Consult the User Documentation for details about what packages are required to run a method. Make note of these.
5) Install required package versions using __renv::install("package@version")__
For example, to install devtools 2.4.4 use __renv::install("devtools@2.4.4")__
6) Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using __sessionInfo()__ — 30/09/2024 at 11:31:14 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 7
Updated At: 2024-09-30T12:00:26.607Z
Content Updated:
id: install-a-method
title: Install a method
content: # Install a method
### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**IDS Users:**
IDS users should follow the IDS guidance for downloading and installing SML methods.
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter: __pip install sml_small__
3. To install a specific version enter: __pip install sml_small==version_number__
For example: __pip install sml_small==1.1.0__
Search "sml_small" on “PyPI · The Python Package Index” for available versions.
4. To use Date Adjustment or Selective Editing run the following: __import sml_small.date_adjustment as date_adjust__ OR __import sml_small.selective_editing as seled__
5. To use Totals and Components or Thousand Pounds run the following: __from sml_small.editing import thousand_pounds__ OR __from sml_small.editing import totals_and_components__
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter __pip install method_name__
For example, the Python version of cell-key-perturbation is installed using __pip install cell_key_perturbation__
3. To install a specific version enter __pip install method_name==version_number__
For example, __pip install cell_key_perturbation==1.1.1__
Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
1) Install the method in your project using __renv::install("method_name")__
For example, if the method was named "aggregate" you would enter __renv::install("aggregate")__
2) To install a specific version use __renv::install("package@version")__
For example: __renv::install("aggregate@1.2.1")__
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
1) Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
2) At the prompt enter __conda create environment_name__
For example, to create and environment called "test_env" you would enter __conda create test_env__
You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
3) Enter __conda activate environment_name__
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment.
To leave the environment at any time enter __conda deactivate__ To come back to your environment later, simply repeat step 3.
5) Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
4) Now, enter __conda install python=version_number__ If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in your environment after successfully installing Python. If not, enter __conda install pip__
5) Install required packages using __pip install package_name==version__.
For example, __pip install pandas==1.5.3__
If you want to check what is currently installed in your environment enter __conda list__
6) If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering __conda deactivate__ from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
1) Enter __pip install spyder-kernels__ (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels.
To do this enter __pip install spyder-kernels=1.9.2__
2) Open Spyder.
3) From the top menu, go to __Tools, Preferences, Python Interpreter__.
4) Select __Use the following Python Interpreter__.
5) Click on the __select file__ icon (to the right of the address bar below).
6) Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter __conda list__ from within your virtual environment. The file path you need will appear at the top of the list.
7) Click on the folder to open your virtual environment and scroll down to find and click on the __python.exe file__.
8) Click __Apply__ and __OK__ to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
9) When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
1) Enter __pip install ipykernel__ (still within Anaconda prompt and your virtual environment).
2) Then enter __python -m ipykernel install --name env_name__
3) Start Jupyter Notebook.
4) From the __Files__ page (you should be here by default) go to ```New```, located in the top right.
5) Click on __New__ and select your virtual environment name.
6) Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
1) Open R Studio and select __File -> New Project__.
2) Select __New Directory -> New Project__ and give your directory a useful name.
3) Tick __Use renv with this project__.
4) Consult the User Documentation for details about what packages are required to run a method. Make note of these.
5) Install required package versions using __renv::install("package@version")__
For example, to install devtools 2.4.4 use __renv::install("devtools@2.4.4")__
6) Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using __sessionInfo()__ — 30/09/2024 at 12:00:26 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 8
Updated At: 2024-09-30T12:39:04.761Z
Content Updated:
id: install-a-method
title: Install a method
content: ### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**IDS Users:**
IDS users should follow the IDS guidance for downloading and installing SML methods.
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter: __pip install sml_small__
3. To install a specific version enter: __pip install sml_small==version_number__
For example: __pip install sml_small==1.1.0__
Search "sml_small" on “PyPI · The Python Package Index” for available versions.
4. To use Date Adjustment or Selective Editing run the following: __import sml_small.date_adjustment as date_adjust__ OR __import sml_small.selective_editing as seled__
5. To use Totals and Components or Thousand Pounds run the following: __from sml_small.editing import thousand_pounds__ OR __from sml_small.editing import totals_and_components__
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter __pip install method_name__
For example, the Python version of cell-key-perturbation is installed using __pip install cell_key_perturbation__
3. To install a specific version enter __pip install method_name==version_number__
For example, __pip install cell_key_perturbation==1.1.1__
Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
1) Install the method in your project using __renv::install("method_name")__
For example, if the method was named "aggregate" you would enter __renv::install("aggregate")__
2) To install a specific version use __renv::install("package@version")__
For example: __renv::install("aggregate@1.2.1")__
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
1) Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
2) At the prompt enter __conda create environment_name__
For example, to create and environment called "test_env" you would enter __conda create test_env__
You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
3) Enter __conda activate environment_name__
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment.
To leave the environment at any time enter __conda deactivate__ To come back to your environment later, simply repeat step 3.
5) Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
4) Now, enter __conda install python=version_number__ If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in your environment after successfully installing Python. If not, enter __conda install pip__
5) Install required packages using __pip install package_name==version__.
For example, __pip install pandas==1.5.3__
If you want to check what is currently installed in your environment enter __conda list__
6) If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering __conda deactivate__ from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
1) Enter __pip install spyder-kernels__ (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels.
To do this enter __pip install spyder-kernels=1.9.2__
2) Open Spyder.
3) From the top menu, go to __Tools, Preferences, Python Interpreter__.
4) Select __Use the following Python Interpreter__.
5) Click on the __select file__ icon (to the right of the address bar below).
6) Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter __conda list__ from within your virtual environment. The file path you need will appear at the top of the list.
7) Click on the folder to open your virtual environment and scroll down to find and click on the __python.exe file__.
8) Click __Apply__ and __OK__ to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
9) When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
1) Enter __pip install ipykernel__ (still within Anaconda prompt and your virtual environment).
2) Then enter __python -m ipykernel install --name env_name__
3) Start Jupyter Notebook.
4) From the __Files__ page (you should be here by default) go to ```New```, located in the top right.
5) Click on __New__ and select your virtual environment name.
6) Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
1) Open R Studio and select __File -> New Project__.
2) Select __New Directory -> New Project__ and give your directory a useful name.
3) Tick __Use renv with this project__.
4) Consult the User Documentation for details about what packages are required to run a method. Make note of these.
5) Install required package versions using __renv::install("package@version")__
For example, to install devtools 2.4.4 use __renv::install("devtools@2.4.4")__
6) Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using __sessionInfo()__ — 30/09/2024 at 12:39:04 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 9
Updated At: 2024-10-01T07:40:29.841Z
Content Updated:
id: install-a-method
title: Install a method
content: ### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**IDS Users:**
IDS users should follow the IDS guidance for downloading and installing SML methods.
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter: __pip install sml_small__
3. To install a specific version enter: __pip install sml_small==version_number__
For example: __pip install sml_small==1.1.0__
Search "sml_small" on “PyPI · The Python Package Index” for available versions.
4. To use Date Adjustment or Selective Editing run the following: __import sml_small.date_adjustment as date_adjust__ OR __import sml_small.selective_editing as seled__
5. To use Totals and Components or Thousand Pounds run the following: __from sml_small.editing import thousand_pounds__ OR __from sml_small.editing import totals_and_components__
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
1. Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
2. From your Python environment enter __pip install method_name__
For example, the Python version of cell-key-perturbation is installed using __pip install cell_key_perturbation__
3. To install a specific version enter __pip install method_name==version_number__
For example, __pip install cell_key_perturbation==1.1.1__
Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
1) Install the method in your project using __renv::install("method_name")__
For example, if the method was named "aggregate" you would enter __renv::install("aggregate")__
2) To install a specific version use __renv::install("package@version")__
For example: __renv::install("aggregate@1.2.1")__
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
1) Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
2) At the prompt enter __conda create environment_name__
For example, to create and environment called "test_env" you would enter __conda create test_env__
You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
3) Enter __conda activate environment_name__
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment.
To leave the environment at any time enter __conda deactivate__ To come back to your environment later, simply repeat step 3.
5) Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
4) Now, enter __conda install python=version_number__ If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in your environment after successfully installing Python. If not, enter __conda install pip__
5) Install required packages using __pip install package_name==version__.
For example, __pip install pandas==1.5.3__
If you want to check what is currently installed in your environment enter __conda list__
6) If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering __conda deactivate__ from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
1) Enter __pip install spyder-kernels__ (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels.
To do this enter __pip install spyder-kernels=1.9.2__
2) Open Spyder.
3) From the top menu, go to __Tools, Preferences, Python Interpreter__.
4) Select __Use the following Python Interpreter__.
5) Click on the __select file__ icon (to the right of the address bar below).
6) Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter __conda list__ from within your virtual environment. The file path you need will appear at the top of the list.
7) Click on the folder to open your virtual environment and scroll down to find and click on the __python.exe file__.
8) Click __Apply__ and __OK__ to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
9) When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
1) Enter __pip install ipykernel__ (still within Anaconda prompt and your virtual environment).
2) Then enter __python -m ipykernel install --name env_name__
3) Start Jupyter Notebook.
4) From the __Files__ page (you should be here by default) go to ```New```, located in the top right.
5) Click on __New__ and select your virtual environment name.
6) Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
1) Open R Studio and select __File -> New Project__.
2) Select __New Directory -> New Project__ and give your directory a useful name.
3) Tick __Use renv with this project__.
4) Consult the User Documentation for details about what packages are required to run a method. Make note of these.
5) Install required package versions using __renv::install("package@version")__
For example, to install devtools 2.4.4 use __renv::install("devtools@2.4.4")__
6) Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using __sessionInfo()__ — 01/10/2024 at 07:40:29 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 10
Updated At: 2024-10-01T09:48:25.020Z
Content Updated:
id: install-a-method
title: Install a method
content: ### This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**IDS Users:**
IDS users should follow the IDS guidance for downloading and installing SML methods.
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
- Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
- From your Python environment enter: __pip install sml_small__
- To install a specific version enter: __pip install sml_small==version_number__, for example: __pip install sml_small==1.1.0__
Search "sml_small" on “PyPI · The Python Package Index” for available versions.
- To use Date Adjustment or Selective Editing run the following: __import sml_small.date_adjustment as date_adjust__ OR __import sml_small.selective_editing as seled__
- To use Totals and Components or Thousand Pounds run the following: __from sml_small.editing import thousand_pounds__ OR __from sml_small.editing import totals_and_components__
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
- Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
- From your Python environment enter __pip install method_name__, for example: the Python version of cell-key-perturbation is installed using __pip install cell_key_perturbation__
- To install a specific version enter __pip install method_name==version_number__, for example: __pip install cell_key_perturbation==1.1.1__
Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
- Install the method in your project using __renv::install("method_name")__, for example, if the method was named "aggregate" you would enter __renv::install("aggregate")__
- To install a specific version use __renv::install("package@version")__, for example: __renv::install("aggregate@1.2.1")__
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
**Managing package dependencies for Python methods using Conda**
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
- Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
- At the prompt enter __conda create environment_name__, for example, to create and environment called "test_env" you would enter __conda create test_env__
You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
- Enter __conda activate environment_name__
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment.
To leave the environment at any time enter __conda deactivate__ To come back to your environment later, simply repeat step 3.
- Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
- Now, enter __conda install python=version_number__ If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in your environment after successfully installing Python. If not, enter __conda install pip__
- Install required packages using __pip install package_name==version__, for example: __pip install pandas==1.5.3__
- If you want to check what is currently installed in your environment enter __conda list__
- If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering __conda deactivate__ from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
**Connecting Spyder to a virtual environment**
- Enter __pip install spyder-kernels__ (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels, e.g., __pip install spyder-kernels=1.9.2__
- Open Spyder.
- From the top menu, go to __Tools, Preferences, Python Interpreter__.
- Select __Use the following Python Interpreter__.
- Click on the __select file__ icon (to the right of the address bar below).
- Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter __conda list__ from within your virtual environment. The file path you need will appear at the top of the list.
- Click on the folder to open your virtual environment and scroll down to find and click on the __python.exe file__.
- Click __Apply__ and __OK__ to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
- When you are done working from this environment, go back to steps 3-6 and select the Default interpreter, instead of your virtual environment.
**Connecting Jupyter Notebook to a virtual environment**
- Enter __pip install ipykernel__ (still within Anaconda prompt and your virtual environment).
- Then enter __python -m ipykernel install --name env_name__
- Start Jupyter Notebook.
- From the __Files__ page (you should be here by default) go to __New__, located in the top right.
- Click on __New__ and select your virtual environment name.
- Your new notebook now will use the Python and package versions installed in this environment.
**Managing package dependencies in R Studio with renv**
- Open R Studio and select __File -> New Project__.
- Select __New Directory -> New Project__ and give your directory a useful name.
- Tick __Use renv with this project__.
- Consult the User Documentation for details about what packages are required to run a method. Make note of these.
- Install required package versions using __renv::install("package@version")__, for example: to install devtools 2.4.4 use __renv::install("devtools@2.4.4")__
- Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using __sessionInfo()__ — 01/10/2024 at 09:48:25 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 11
Updated At: 2024-10-01T10:11:08.612Z
Content Updated:
id: install-a-method
title: Install a method
content: # This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**IDS Users:**
IDS users should follow the IDS guidance for downloading and installing SML methods.
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
- Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
- From your Python environment enter: __pip install sml_small__
- To install a specific version enter: __pip install sml_small==version_number__, for example: __pip install sml_small==1.1.0__
Search "sml_small" on “PyPI · The Python Package Index” for available versions.
- To use Date Adjustment or Selective Editing run the following: __import sml_small.date_adjustment as date_adjust__ OR __import sml_small.selective_editing as seled__
- To use Totals and Components or Thousand Pounds run the following: __from sml_small.editing import thousand_pounds__ OR __from sml_small.editing import totals_and_components__
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
- Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
- From your Python environment enter __pip install method_name__, for example: the Python version of cell-key-perturbation is installed using __pip install cell_key_perturbation__
- To install a specific version enter __pip install method_name==version_number__, for example: __pip install cell_key_perturbation==1.1.1__
Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
- Install the method in your project using __renv::install("method_name")__, for example, if the method was named "aggregate" you would enter __renv::install("aggregate")__
- To install a specific version use __renv::install("package@version")__, for example: __renv::install("aggregate@1.2.1")__
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
### Managing package dependencies for Python methods using Conda
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas as Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
- Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
- At the prompt enter __conda create environment_name__, for example, to create and environment called "test_env" you would enter __conda create test_env__
You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
- Enter __conda activate environment_name__
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment.
To leave the environment at any time enter __conda deactivate__ To come back to your environment later, simply repeat step 3.
- Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
- Now, enter __conda install python=version_number__ If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in your environment after successfully installing Python. If not, enter __conda install pip__
- Install required packages using __pip install package_name==version__, for example: __pip install pandas==1.5.3__
- If you want to check what is currently installed in your environment enter __conda list__
- If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering __conda deactivate__ from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
## Connecting Spyder to a virtual environment
- Enter __pip install spyder-kernels__ (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels, e.g., __pip install spyder-kernels=1.9.2__
- Open Spyder.
- From the top menu, go to __Tools -> Preferences -> Python Interpreter__.
- Select __Use the following Python Interpreter__.
- Click on the __select file__ icon (to the right of the address bar below).
- Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter __conda list__ from within your virtual environment. The file path you need will appear at the top of the list.
- Click on the folder to open your virtual environment and scroll down to find and click on the __python.exe file__.
- Click __Apply__ and __OK__ to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
- When you are done working from this environment, go back to __Tools -> Preferences-> Python Interpreter__, and select the __Default interpreter__, instead of your virtual environment.
## Connecting Jupyter Notebook to a virtual environment
- Enter __pip install ipykernel__ (still within Anaconda prompt and your virtual environment).
- Then enter __python -m ipykernel install --name env_name__
- Start Jupyter Notebook.
- From the __Files__ page (you should be here by default) go to __New__, located in the top right.
- Click on __New__ and select your virtual environment name.
- Your new notebook now will use the Python and package versions installed in this environment.
## Managing package dependencies in R Studio with renv
- Open R Studio and select __File -> New Project__.
- Select __New Directory -> New Project__ and give your directory a useful name.
- Tick __Use renv with this project__.
- Consult the User Documentation for details about what packages are required to run a method. Make note of these.
- Install required package versions using __renv::install("package@version")__, for example: to install devtools 2.4.4 use __renv::install("devtools@2.4.4")__
- Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using __sessionInfo()__ — 01/10/2024 at 10:11:08 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 12
Updated At: 2024-10-02T12:00:57.282Z
Content Updated:
id: install-a-method
title: Install a method
content: # This page covers the following topics:
- How to install date adjustment, selective editing, thousand pounds, or totals and components
- How to install other Python methods
- How to install R methods
- Managing package dependencies with SML methods
**IDS Users:**
IDS users should follow the IDS guidance for downloading and installing SML methods.
**Using a method:**
Information on how to use individual methods can be found in the User Documentation.
You can access the User Documentation from the Method Summary which you find by clicking on the method name in the Method Catalogue.
**Package dependencies:**
SML methods may require installation of specific versions of Python, R or essential packages. To avoid changing installations in your base environment, we recommend creating a virtual environment (Python) or using renv in an R Studio project.
If you’d like further guidance on this, please see “Managing package dependencies with SML methods” below.
## How to install date adjustment, selective editing, thousand pounds, or totals and components
Once you have set up the virtual environment and connected to your IDE (if using), follow the steps below.
- Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
- From your Python environment enter __pip install sml_small__
- To install a specific version enter __pip install sml_small==version_number__, for example: __pip install sml_small==1.1.0__
Search "sml_small" on “PyPI · The Python Package Index” for available versions.
- To use Date Adjustment or Selective Editing run the following: __import sml_small.date_adjustment as date_adjust__ OR __import sml_small.selective_editing as seled__
- To use Totals and Components or Thousand Pounds run the following: __from sml_small.editing import thousand_pounds__ OR __from sml_small.editing import totals_and_components__
The methods are now available for you to use.
## Installing other Python methods
Once you have set up the virtual environment and connected to an IDE (if using), follow the steps below.
- Check you have installed the correct versions of Python and required packages such as Pandas (see User Documentation).
- From your Python environment enter __pip install package_name__, for example: the Python version of cell-key-perturbation is installed using __pip install cell_key_perturbation__
- To install a specific version enter __pip install package_name==version_number__, for example: __pip install cell_key_perturbation==1.1.1__
Search the method name on “PyPI · The Python Package Index” for available versions.
The method is now available for you to import.
## Installing R methods
Once you have created an R Studio project using renv to install required packages, you are ready to download the method package. For more guidance on using renv please see 'Managing package dependencies in R Studio with renv' below.
- Install the method package in your project using __renv::install("package_name")__, for example, if the method package was named "aggregate" you would enter __renv::install("aggregate")__
- To install a specific version use __renv::install("package_name@version")__, for example: __renv::install("aggregate@1.2.1")__
The method is now available for you to load.
## Managing package dependencies with SML methods
This section covers the following topics:
- Managing package dependencies for Python methods using Conda
- Connecting Spyder to a virtual environment
- Connecting Jupyter Notebook to a virtual environment
- Managing package dependencies in R Studio using renv
### Managing package dependencies for Python methods using Conda
Virtual environments allow you to install specific versions of software in a contained environment. For example, you can install an older version of Python in a virtual environment but continue to use the most recent version of Python in your default environment.
Please refer to the method User Documentation for information about which version of Python and packages like Pandas or Numpy you need to use for a specific method. You will need this information to correctly set up your environment.
This guidance uses conda to create a virtual environment on your device. Conda is an open-source environment management system that allows you to create virtual environments.
- Search Anaconda Prompt on your device and click to open it (note, this will only work if you have already installed Anaconda or Miniconda).
- At the prompt enter __conda create environment_name__, for example, to create and environment called "test_env" you would enter __conda create test_env__
You should name your environment something that will help you remember what it’s used for. If prompted, enter 'y' to proceed.
- Enter __conda activate environment_name__
You should now see your environment name to the left of the C:/ prompt. This shows you are working within the virtual environment.
To leave the environment at any time enter __conda deactivate__ To come back to your environment later, enter __conda activate environment_name__, as before.
- Consult the User Documentation to identify what version of Python is required and to see what packages (and package versions) are needed. Make note of these.
- Now, enter __conda install python=version_number__ If prompted, enter 'y' to proceed.
Note, Python versions > 3.4 automatically install pip, so that will likely be available to you in your environment after successfully installing Python. If not, enter __conda install pip__
- Install required packages using __pip install package_name==version__, for example: __pip install pandas==1.5.3__
- If you want to check what is currently installed in your environment enter __conda list__
- If you would like to use the methods from within an IDE like Visual Studio, Spyder, or Jupyter Notebook, then you will need to install the kernels for these in the virtual environment and connect them to your session (see further guidance below)
Your virtual environment is now set up and ready for you to install your chosen SML method. Remember to deactivate your environment when you’re done setting it up by entering __conda deactivate__ from the Anaconda prompt window.
The environment does not need to be activated here for you to use it with Spyder or Jupyter Notebook, providing you have followed the steps to connect the environment to these.
## Connecting Spyder to a virtual environment
- Enter __pip install spyder-kernels__ (still within Anaconda prompt and your virtual environment). If you have installed an older version of python (e.g., 3.7.1 for use with sml_small) then you need to specify an older version of spyder-kernels, e.g., __pip install spyder-kernels=1.9.2__
- Open Spyder.
- From the top menu, go to __Tools -> Preferences -> Python Interpreter__.
- Select __Use the following Python Interpreter__.
- Click on the __select file__ icon (to the right of the address bar below).
- Navigate to the location of your virtual environment.
If you’re unsure where this is, go to your Anaconda prompt window and enter __conda list__ from within your virtual environment. The file path you need will appear at the top of the list.
- Click on the folder to open your virtual environment and scroll down to find and click on the __python.exe file__.
- Click __Apply__ and __OK__ to close the Preferences window.
You should see your virtual environment name and the python version it’s running on the bottom of the Spyder window.
- When you are done working from this environment, go back to __Tools -> Preferences-> Python Interpreter__, and select the __Default interpreter__, instead of your virtual environment.
## Connecting Jupyter Notebook to a virtual environment
- Enter __pip install ipykernel__ (still within Anaconda prompt and your virtual environment).
- Then enter __python -m ipykernel install --name env_name__
- Start Jupyter Notebook.
- From the __Files__ page (you should be here by default) go to __New__, located in the top right.
- Click on __New__ and select your virtual environment name.
Your new notebook now will use the Python and package versions installed in this environment.
## Managing package dependencies in R Studio with renv
- Open R Studio and select __File -> New Project__.
- Select __New Directory -> New Project__ and give your directory a useful name.
- Tick __Use renv with this project__.
- Consult the User Documentation for details about what packages are required to run a method. Make note of these.
- Install required package versions using __renv::install("package@version")__, for example: to install devtools 2.4.4 use __renv::install("devtools@2.4.4")__
- Download and run the chosen SML method within this project environment (see "Installing R Methods" above for more guidance).
Using renv will make specified versions of packages available for this project, but will not affect versions installed outside the project.
You can verify which versions are installed using __sessionInfo()__ — 02/10/2024 at 12:00:57 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-10-07T07:16:09.663Z
Content Updated: entry 37TwJlBJgUvgCRJ7Tq30pt has been deleted.
id: {"en-US":"provide-feedback"}
title: {"en-US":"Provide feedback"}
content: {"en-US":"Please email [smlhelp@ons.gov.uk](mailto:smlhelp@ons.gov.uk)."}— 07/10/2024 at 07:16:09 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-10-07T07:16:37.121Z
Content Updated: entry 76zlJrhByCFNZlbKVA5AtB has been deleted.
id: {"en-US":"expert-groups"}
title: {"en-US":"Get information on expert groups"}
content: {"en-US":"Please visit [Expert groups](https://analysisfunction.civilservice.gov.uk/government-statistical-service-and-statistician-group/gss-support/methodology/information-on-specific-methods/)."}— 07/10/2024 at 07:16:37 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-10-07T07:17:10.991Z
Content Updated: entry 3cCYV7jPz7iisw87gZvGgK has been deleted.
id: {"en-US":"version-methods"}
title: {"en-US":"How the methods are versioned"}
content: {"en-US":"The Statistical Methods Library (SML) and the associated methods are versioned using semantic versioning comprised of a major, minor and patch release version separated by full stops (e.g 3.5.4). \n\nThere are separate releases of the libraries based on the method implementation, python/pandas or python/pySpark, and each library uses semantic versioning.\n\nThe major version is incremented (and minor and patch releases reset to zero) when backward incompatible changes are made (e.g renaming a pre-existing method function name).\n\nThe minor version is incremented (and the patch version reset to zero) when new features are added without breaking backward compatibility (e.g addition of a new statistical method)\n\nThe patch release version is incremented when backward compatible bugfixes are implemented.\n\nConsult the [SML python/pySpark release history in Github](https://github.com/ONSdigital/statistical-methods-library/releases) for the current and past releases for the python/pySpark methods and the [associated python/pySpark release notes](https://github.com/ONSdigital/statistical-methods-library/tree/main/docs/release-notes) to understand the changes between releases.\n\nConsult the [SML python/pandas release history in Github](https://github.com/ONSdigital/sml-python-small/releases) for the current and past releases for the python/pandas methods and the [associated python/pandas release notes](https://github.com/ONSdigital/sml-python-small/tree/main/docs/release-notes) to understand the changes between releases.\n\nFor further information about semantic versioning read the recommended [semantic versioning guidance](https://semver.org)."}— 07/10/2024 at 07:17:10 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-10-07T07:17:35.675Z
Content Updated: entry 5WaengmajUsGSEbXHxhu1r has been deleted.
id: {"en-US":"github"}
title: {"en-US":"Using GitHub"}
content: {"en-US":"This section contains useful links for anyone unfamiliar with Github, it provides some pointers to documentation that will help you use and navigate the system.\n\nFor general guidance consult the [Github documentation](https://docs.github.com/en) in the first instance\n#### Releases\n\nConsult the [Github guidance on navigating releases](https://docs.github.com/en/repositories/releasing-projects-on-github/comparing-releases) to get an understanding of identifying where the release information can be found and how to search for a specific release.\n\nView the [Github release comparison documentation](https://docs.github.com/en/repositories/releasing-projects-on-github/comparing-releases) to get information on comparing different release versions and understanding how to use Github to determine the changes between releases.\n"}— 07/10/2024 at 07:17:35 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-10-07T07:17:52.483Z
Content Updated: entry 3vD2v4Ma4DoxClPZgeQxdE has been deleted.
id: {"en-US":"report-bug"}
title: {"en-US":"Report a defect or bug"}
content: {"en-US":"Please email [smlhelp@ons.gov.uk](mailto:smlhelp@ons.gov.uk)."}— 07/10/2024 at 07:17:52 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-10-07T07:18:09.843Z
Content Updated: entry 6WQ6Wtug7CnYQigqvDNH9S has been deleted.
id: {"en-US":"coding-standards"}
title: {"en-US":"Coding standards"}
content: {"en-US":"Code will be written to internationally recognised standards and according to principles set out in these links:-\n\n* [Government Digital Service](https://gds-way.cloudapps.digital/#the-gds-way/)\n* [ONS Quality Assurance for Code and Analysis (Duck book)](https://best-practice-and-impact.github.io/qa-of-code-guidance/intro.html)\n* [ONS Digital Software Engineering Standards](https://github.com/ONSdigital/software-engineer-community/blob/master/standards/README.md)\n\nFor Python, this means it will adhere to the [PEP8 Python style guide](https://peps.python.org/pep-0008/) and the [GDS Python style guide.](https://gds-way.cloudapps.digital/manuals/programming-languages/python/python.html#content)\n"}— 07/10/2024 at 07:18:09 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-10-07T07:18:21.402Z
Content Updated: entry 3nzjuL0uUBerLb5tQYGuq8 has been deleted.
id: {"en-US":"view-methods"}
title: {"en-US":"Find and view methods"}
content: {"en-US":"You can find and view methods within the [methods catalogue](/methods)\n\nWhere available, links to GitHub for source code and specifications will be on the method's details page.\n"}— 07/10/2024 at 07:18:21 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 2
Updated At: 2024-10-07T07:21:02.106Z
Content Updated:
id: troubleshooting
title: Troubleshooting
content: As issues using our methods are raised we will update this section with relevant information to resolve the issue.
#### Pip install issues:
Please note all commands in this section should be run in either a terminal(Linux/Mac) or a command prompt(Windows).
If you get an error when running pip install and can't immediately see the error try to add the --verbose flag to the
command to see if that offers any more useful information, this can be done by running the command like so:
"pip install {package} --verbose"
If multiple versions of python are installed on your machine it may also be the case that you need to run the command as
such:
"pip3 install {package}"
Some other common errors include:
1. Package name is incorrect, double check that the file path and name that you are giving to the command is correct.
2. If you have an outdated version of pip this can occasionally cause an issue, try running the below commands:
"python -m pip install --update pip"
"python -m pip install --update wheel distutils"
3. Permission errors, options here include running as admin/sudo where allowed or run:
"pip install {package} --user"
This will run pip install defaulting the install location to the users home directory, as opposed to a system directory
like it normally would.
4. Failure during install, on Windows in particular a library can have issues installing due to packages it relies on,
try running:
"pip install {package} --prefer-binary"
This will cause pip to try install binary packages where possible as windows can sometimes throw errors compiling from
source
#### Alternatives to pip for installation:
Some users may be in the scenario that due to user privileges that they are not allowed to use pip on their machines.
As an alternative to the .whl file, within a given release there will also be a tar.gz file in the format of
{LibraryName-version.tar.gz}. Download this file and extract it:
- On a Unix system (linux/OSX) run the following command in a terminal: "tar -xf archive.tar.gz". Substituting archive.tar.gz
with the path to the tar.gz file downloaded
- On a Windows system open the file within 7Zip or a similar file archive program and follow the interface to extract the
folder.
Navigate to the extracted folder within a terminal or command prompt and run the following command:
"python setup.py install".
Once this completes the library should now be installed on your machine. Please note on the above command to ensure
that you use the correct python command. Particularly for unix users, python V3.0 and higher will often be run as python3
within the terminal. If the previous command failed, it may be that you need to instead substitute it with:
"python3 setup.py install".
Alternatively on Windows machine python may sometimes be installed as py so:
"py setup.py install".— 07/10/2024 at 07:21:02 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreStructure
Revision: 3
Updated At: 2024-10-07T07:57:23.419Z
Content Updated:
structure: {"categories":[{"name":"access","label":"Accessing Methods","subcategories":[{"name":"install-a-method","label":"Install a method"}]},{"name":"feedback","label":"Feedback","subcategories":[{"name":"get-in-touch","label":"Get in touch"}]},{"name":"support","label":"Support","subcategories":[{"name":"troubleshooting","label":"Troubleshooting (in development)"}]}]}— 07/10/2024 at 07:57:23 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 2
Updated At: 2024-10-07T08:09:33.202Z
Content Updated:
id: get-in-touch
title: Get in touch
content: If you'd like to speak to someone on the SML team, please email [smlhelp@ons.gov.uk](mailto:smlhelp@ons.gov.uk).
If you need help installing or using a method, make sure you've consulted the Help Centre "Install a Method" page and have read the relevant user documentation, which you can find with the Method Summary. — 07/10/2024 at 08:09:33 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreInformation
Revision: 3
Updated At: 2024-10-07T11:25:44.443Z
Content Updated:
id: get-in-touch
title: Get in touch
content: If you'd like to speak to someone on the SML team, please email [smlhelp@ons.gov.uk](mailto:smlhelp@ons.gov.uk).
Need help installing a method? Make sure you've consulted the Help Centre "Install a Method" page.
Need help using a method? Make sure you have read the user documentation, which you can find by clicking on the method name in the Method Catalogue.
If you still need assistance or if you have suggestions or other feedback, please do get in touch with us using the email above.— 07/10/2024 at 11:25:44 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreStructure
Revision: 4
Updated At: 2024-10-07T11:38:53.079Z
Content Updated:
structure: {"categories":[{"name":"access","label":"Accessing Methods","subcategories":[{"name":"install-a-method","label":"Install a method"}]},{"name":"feedback","label":"Feedback","subcategories":[{"name":"get-in-touch","label":"Get in touch"}]},{"name":"support","label":"Support","subcategories":[{"name":"troubleshooting","label":"Troubleshooting"}]}]}— 07/10/2024 at 11:38:53 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: about
Revision: 2
Updated At: 2024-10-07T12:06:36.533Z
Content Updated:
about: # About the Statistical Methods Library
## What is the Statistical Methods Library?
The Statistical Methods Library (SML) is a central library for approved statistical methods for use by a wide range of users.
## SML mission
Our mission is to provide a central resource for consistent and reproducible statistical methods and documentation, including a governance process to facilitate quality assurance, adding, and updating of methods. The SML will allow methods and best practice to be shared across ONS, other government departments and wider.
## Current objectives
- To act as a shared single point of storage and reference for best practice methods and method documentation
- To ensure methods are available across ONS, other government departments and a wider user base, through a public portal and the Integrated Data Service (IDS).
- To ensure that methods for official statistical outputs are fully transparent.
- To provide clear governance for method quality assurance. — 07/10/2024 at 12:06:36 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: about
Revision: 3
Updated At: 2024-10-07T12:42:20.717Z
Content Updated:
about: # About the Statistical Methods Library
## What is the Statistical Methods Library?
The Statistical Methods Library (SML) is a central library for approved statistical methods, accessible to a wide range of users.
## SML mission
Our mission is to provide a central resource for consistent and reproducible statistical methods and documentation, including a governance process to facilitate quality assurance and adding/updating methods. The SML will allow methods and best practice to be shared across ONS, other government departments and wider.
## Current objectives
- To act as a shared single point of storage and reference for best practice methods and method documentation
- To ensure methods are available across ONS, other government departments and a wider user base, through a public portal and the Integrated Data Service (IDS).
- To ensure that methods for official statistical outputs are fully transparent.
- To provide clear governance for method quality assurance. — 07/10/2024 at 12:42:20 PM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 2
Updated At: 2024-10-09T11:22:30.231Z
Content Updated:
id: sml
term: SML
meaning: The Statistical Methods Library, a centralised collection of methods approved for use by the ONS. Comprises a code library, accessed through GitHub, and a collection of associated documentation, accessed via the SML Portal.— 09/10/2024 at 11:22:30 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-10-10T09:32:46.808Z
Content Updated:
id: totals-and-components
name: Totals and Components
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: The automatic editing method for totals and components correction is currently used in ONS business surveys to ensure fixed relationships between variables are satisfied. For example, when a total (e.g., total employment) is collected along with the component breakdown (e.g., full-time male, full-time female, part-time male, part-time female). The primary use of the method is to automatically detect and correct errors in respondent data where fixed relationships have not been satisfied to improve the efficiency of the editing process, reduce the burden on respondents and survey validators and improve overall data quality. This method can also be used to ensure fixed relationships between variables are satisfied in other data types such as imputed data to improve overall data quality.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.1.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.1.0
codeLink: https://github.com/ONSdigital/sml-python-small/tree/main/sml_small/editing/totals_and_components/totals_and_components.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/totals_components/totals_components.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md— 10/10/2024 at 09:32:46 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-10-10T09:38:17.592Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.0.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.0.0
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 10/10/2024 at 09:38:17 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-10-10T10:33:32.286Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.0.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.0.0
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 10/10/2024 at 10:33:32 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 4
Updated At: 2024-10-11T11:21:32.705Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.1.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.1.0
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 11/10/2024 at 11:21:32 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-10-11T11:33:19.756Z
Content Updated:
id: selective-editing
name: Selective Editing
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: Selective Editing is an internationally recognised editing method where potential errors are prioritised according to their expected effect on key outputs. Only respondents that are having an impact on published estimates will be recontacted.
Selective Editing works by assigning a score to each important variable for a business where, the score reflects the impact that editing the respondent will have on the estimates. Only contributors with a high score are checked, low scoring contributors pass through unchecked.
To install this method, please see Help Centre - [Install a method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/selective-editing/selective_editing.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.1.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.1.0
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/selective_editing.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/selective_editing/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/selective-editing/selective_editing.md— 11/10/2024 at 11:33:19 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-10-11T11:42:43.777Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.1.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.1.0
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/thousand_pounds_correction/thousand_pounds_correction.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md— 11/10/2024 at 11:42:43 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-10-11T11:50:25.549Z
Content Updated:
id: totals-and-components
name: Totals and Components
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: The automatic editing method for totals and components correction is currently used in ONS business surveys to ensure fixed relationships between variables are satisfied. For example, when a total (e.g., total employment) is collected along with the component breakdown (e.g., full-time male, full-time female, part-time male, part-time female). The primary use of the method is to automatically detect and correct errors in respondent data where fixed relationships have not been satisfied to improve the efficiency of the editing process, reduce the burden on respondents and survey validators and improve overall data quality. This method can also be used to ensure fixed relationships between variables are satisfied in other data types such as imputed data to improve overall data quality.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.1.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.1.0
codeLink: https://github.com/ONSdigital/sml-python-small/tree/main/sml_small/editing/totals_and_components/totals_and_components.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/totals_components/totals_components.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md— 11/10/2024 at 11:50:25 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-10-18T13:47:30.515Z
Content Updated:
id: metadata
term: Metadata
meaning: 'Data about data', giving information about a product itself, as opposed to its contents. In the case of the SML, metadata includes, for example: when a method was onboarded; its development status; by whom it was developed; whether or not it is approved for use in official outputs; etc.— 18/10/2024 at 01:47:30 PM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 2
Updated At: 2024-10-18T13:47:27.504Z
Content Updated: entry 1blaUAHFEY66NrdCtomp9k has been deleted.
id: {"en-US":"pypi"}
term: {"en-US":"PyPI"}
meaning: {"en-US":"The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software."}
relatedlist: {"en-US":["SML","Package","Repository"]}
externalLinkTextShort: {"en-US":"ONS-SML user profile on PyPI"}
externalUrl: {"en-US":"https://pypi.org/user/ons-sml/"}— 18/10/2024 at 01:47:27 PM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 2
Updated At: 2024-10-18T13:47:23.506Z
Content Updated: entry 3MHWfCcqLpDHbc7NLLuNh6 has been deleted.
id: {"en-US":"statistical_methods"}
term: {"en-US":"Statistical Methods"}
meaning: {"en-US":"Mathematical techniques used to transform, analyse and manipulate data that fall under the remit of one of the expert groups within the Methodology and Quality Directorate of the ONS."}— 18/10/2024 at 01:47:23 PM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-10-18T13:47:30.515Z
Content Updated:
id: metadata
term: Metadata
meaning: 'Data about data', giving information about a product itself, as opposed to its contents. In the case of the SML, metadata includes, for example: when a method was onboarded; its development status; by whom it was developed; whether or not it is approved for use in official outputs; etc.— 18/10/2024 at 01:47:30 PM —
Editor: Jason Bell
Environment: preprod
Content Type: helpCentreMethodRequest
Revision: 1
Updated At: 2024-10-18T13:47:31.497Z
Content Updated:
name: Help Centre Method Request
internalUserTitle: I am an ONS internal user
internalUserContent: Method requests can be raised to:
* Have new methods developed for the Statistical Methods Library.
* Add existing code to the Statistical Methods Library.
* Change existing methods held within the Statistical Methods Library.
If you would like to make a suggestion for a new method, or to provide feedback about an existing method, please do so by emailing [smlhelp@ons.gov.uk](mailto:smlhelp@ons.gov.uk).
externalUserTitle: I am a user from outside the ONS
externalUserContent: Currently we do not accept formal method submissions or change requests from external users. In future, we will accept certain method requests via the [Integrated Data Service](https://integrateddataservice.gov.uk/).— 18/10/2024 at 01:47:31 PM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-10-18T13:47:31.499Z
Content Updated:
id: community_methods
term: Community Methods
meaning: Code developed by a wider community of analysts. These methods are held in a community code repository to ensure there is a clear distinction in terms of maintenance of the code. After sign-off, any changes to methods held in the SML community repository must be made via the SML governance process to ensure that specifications and documentation are kept up to date and users are informed of changes to the code. The developers of the code would be accountable for making any updates to the code.— 18/10/2024 at 01:47:31 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-10-31T10:29:55.018Z
Content Updated:
id: selective-editing
name: Selective Editing
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: Selective Editing is an internationally recognised editing method where potential errors are prioritised according to their expected effect on key outputs. Only respondents that are having an impact on published estimates will be recontacted.
Selective Editing works by assigning a score to each important variable for a business where, the score reflects the impact that editing the respondent will have on the estimates. Only contributors with a high score are checked, low scoring contributors pass through unchecked.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/selective_editing.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/selective_editing/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/selective-editing/selective_editing.md— 31/10/2024 at 10:29:55 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-10-31T10:30:33.378Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 31/10/2024 at 10:30:33 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-10-31T10:33:35.754Z
Content Updated:
id: totals-and-components
name: Totals and Components
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for totals and components correction is currently used in ONS business surveys to ensure fixed relationships between variables are satisfied. For example, when a total (e.g., total employment) is collected along with the component breakdown (e.g., full-time male, full-time female, part-time male, part-time female). The primary use of the method is to automatically detect and correct errors in respondent data where fixed relationships have not been satisfied to improve the efficiency of the editing process, reduce the burden on respondents and survey validators and improve overall data quality. This method can also be used to ensure fixed relationships between variables are satisfied in other data types such as imputed data to improve overall data quality.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
codeLink: https://github.com/ONSdigital/sml-python-small/tree/main/sml_small/editing/totals_and_components/totals_and_components.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/totals_components/editing_and_imputation/totals_components.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md— 31/10/2024 at 10:33:35 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-10-31T10:34:17.012Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/thousand_pounds_correction/thousand_pounds_correction.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md— 31/10/2024 at 10:34:17 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: heroHomePage
Revision: 2
Updated At: 2024-10-31T10:34:52.313Z
Content Updated:
heroTitle: A library for statistical code approved by the ONS
heroButton: Find methods— 31/10/2024 at 10:34:52 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: helpCentreMethodRequest
Revision: 3
Updated At: 2024-10-31T10:35:36.690Z
Content Updated: entry 3yPRVuZ5tvZdP09NLOauEj has been deleted.
name: {"en-US":"Help Centre Method Request"}
internalUserTitle: {"en-US":"I am an ONS internal user"}
internalUserContent: {"en-US":"Method requests can be raised to:\n\n* Have new methods developed for the Statistical Methods Library.\n* Add existing code to the Statistical Methods Library.\n* Change existing methods held within the Statistical Methods Library.\n\nIf you would like to make a suggestion for a new method, or to provide feedback about an existing method, please do so by emailing [smlhelp@ons.gov.uk](mailto:smlhelp@ons.gov.uk)."}
externalUserTitle: {"en-US":"I am a user from outside the ONS"}
externalUserContent: {"en-US":"Currently we do not accept formal method submissions or change requests from external users. In future, we will accept certain method requests via the [Integrated Data Service](https://integrateddataservice.gov.uk/)."}— 31/10/2024 at 10:35:36 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-10-31T12:49:48.373Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/thousand_pounds_correction/thousand_pounds_correction.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md— 31/10/2024 at 12:49:48 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-10-31T12:59:07.451Z
Content Updated:
id: selective-editing
name: Selective Editing
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: Selective Editing is an internationally recognised editing method where potential errors are prioritised according to their expected effect on key outputs. Only respondents that are having an impact on published estimates will be recontacted.
Selective Editing works by assigning a score to each important variable for a business where, the score reflects the impact that editing the respondent will have on the estimates. Only contributors with a high score are checked, low scoring contributors pass through unchecked.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/selective_editing.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/selective_editing/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/selective-editing/selective_editing.md— 31/10/2024 at 12:59:07 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-10-31T12:59:55.947Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 31/10/2024 at 12:59:55 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-10-31T13:01:12.781Z
Content Updated:
id: totals-and-components
name: Totals and Components
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for totals and components correction is currently used in ONS business surveys to ensure fixed relationships between variables are satisfied. For example, when a total (e.g., total employment) is collected along with the component breakdown (e.g., full-time male, full-time female, part-time male, part-time female). The primary use of the method is to automatically detect and correct errors in respondent data where fixed relationships have not been satisfied to improve the efficiency of the editing process, reduce the burden on respondents and survey validators and improve overall data quality. This method can also be used to ensure fixed relationships between variables are satisfied in other data types such as imputed data to improve overall data quality.
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/tree/main/sml_small/editing/totals_and_components/totals_and_components.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/totals_components/editing_and_imputation/totals_components.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md— 31/10/2024 at 01:01:12 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 5
Updated At: 2024-11-05T14:45:25.635Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.0.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.0.0
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 05/11/2024 at 02:45:25 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 5
Updated At: 2024-11-05T14:45:25.635Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python/Pandas
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.0.0
releaseLink: https://github.com/ONSdigital/sml-python-small/releases/tag/1.0.0
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 05/11/2024 at 02:45:25 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 6
Updated At: 2024-11-06T08:55:27.129Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.1
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 06/11/2024 at 08:55:27 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 7
Updated At: 2024-11-06T12:15:44.747Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.2
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 06/11/2024 at 12:15:44 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-11-06T12:24:17.693Z
Content Updated:
id: selective-editing
name: Selective Editing
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: Selective Editing is an internationally recognised editing method where potential errors are prioritised according to their expected effect on key outputs. Only respondents that are having an impact on published estimates will be recontacted.
Selective Editing works by assigning a score to each important variable for a business where, the score reflects the impact that editing the respondent will have on the estimates. Only contributors with a high score are checked, low scoring contributors pass through unchecked.
To install this method, please see Help Centre - [Install a method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/selective-editing/selective_editing.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.2
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/selective_editing.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/selective_editing/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/selective-editing/selective_editing.md— 06/11/2024 at 12:24:17 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-11-06T12:26:57.113Z
Content Updated:
id: totals-and-components
name: Totals and Components
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for totals and components correction is currently used in ONS business surveys to ensure fixed relationships between variables are satisfied. For example, when a total (e.g., total employment) is collected along with the component breakdown (e.g., full-time male, full-time female, part-time male, part-time female). The primary use of the method is to automatically detect and correct errors in respondent data where fixed relationships have not been satisfied to improve the efficiency of the editing process, reduce the burden on respondents and survey validators and improve overall data quality. This method can also be used to ensure fixed relationships between variables are satisfied in other data types such as imputed data to improve overall data quality.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.2
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/tree/main/sml_small/editing/totals_and_components/totals_and_components.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/totals_components/editing_and_imputation/totals_components.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md— 06/11/2024 at 12:26:57 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 7
Updated At: 2024-11-06T12:31:29.568Z
Content Updated: entry 4GwadO5feNlF2biBAAZbfw has been deleted.
id: {"en-US":"totals-and-components"}
name: {"en-US":"Totals and Components"}
theme: {"en-US":"Editing"}
expertGroup: {"en-US":"Editing & Imputation"}
language: {"en-US":"Python"}
author: {"en-US":"ONS"}
description: {"en-US":"The automatic editing method for totals and components correction is currently used in ONS business surveys to ensure fixed relationships between variables are satisfied. For example, when a total (e.g., total employment) is collected along with the component breakdown (e.g., full-time male, full-time female, part-time male, part-time female). The primary use of the method is to automatically detect and correct errors in respondent data where fixed relationships have not been satisfied to improve the efficiency of the editing process, reduce the burden on respondents and survey validators and improve overall data quality. This method can also be used to ensure fixed relationships between variables are satisfied in other data types such as imputed data to improve overall data quality.\n\nTo install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).\n\nTo use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md)."}
contactDetails: {"en-US":"smlhelp@ons.gov.uk"}
methodReady: {"en-US":true}
releaseVersion: {"en-US":"v1.2.2"}
releaseLink: {"en-US":"https://github.com/ONSdigital/sml-python-small/releases"}
codeLink: {"en-US":"https://github.com/ONSdigital/sml-python-small/tree/main/sml_small/editing/totals_and_components/totals_and_components.py"}
specificationLink: {"en-US":"https://github.com/ONSdigital/Statistical-Method-Specifications/blob/totals_components/editing_and_imputation/totals_components.md"}
userDocumentationLink: {"en-US":"https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md"}— 06/11/2024 at 12:31:29 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-11-06T12:32:54.156Z
Content Updated:
id: totals-and-components
name: Totals and Components
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for totals and components correction is currently used in ONS business surveys to ensure fixed relationships between variables are satisfied. For example, when a total (e.g., total employment) is collected along with the component breakdown (e.g., full-time male, full-time female, part-time male, part-time female). The primary use of the method is to automatically detect and correct errors in respondent data where fixed relationships have not been satisfied to improve the efficiency of the editing process, reduce the burden on respondents and survey validators and improve overall data quality. This method can also be used to ensure fixed relationships between variables are satisfied in other data types such as imputed data to improve overall data quality.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.2
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/tree/main/sml_small/editing/totals_and_components/totals_and_components.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/totals_components/totals_components.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/totals-and-components/totals-and-components.md— 06/11/2024 at 12:32:54 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-11-06T12:43:14.353Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.2
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/thousand_pounds_correction/thousand_pounds_correction.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md— 06/11/2024 at 12:43:14 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-11-06T12:56:58.452Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases/tag/v2.0.0
codeLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py
userDocumentationLink: https://github.com/ONSdigital/cell-key-perturbation/blob/main/documentation/SML_UserDocs_CellKeyPerturbation_Py.md— 06/11/2024 at 12:56:58 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 8
Updated At: 2024-11-06T15:03:28.465Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.2
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 06/11/2024 at 03:03:28 PM —
Editor: Joseph Percival
Environment: preprod
Content Type: accessibilityPage
Revision: 2
Updated At: 2024-11-06T16:49:16.636Z
Content Updated:
title: Accessibility Statement
mainContent: # Accessibility Statement
This accessibility statement applies to the Office for National Statistics (ONS) Statistical Methods Library.
We want as many people as possible to be able to use the website. If you have a disability, [AbilityNet](https://mcmw.abilitynet.org.uk/) has advice on making your device easier to use.
## How accessible the website is
We are aware that some parts of this website are not fully accessible, and we are working to improve this.
## How to request content in an accessible format
If you need information from the website in a different format please contact us at [SMLHelp@ons.gov.uk](mailto:SMLHelp@ons.gov.uk) and tell us:
* The web address (URL).
* Your name and email address.
* The format you require.
We’ll reply within ten days to let you know when we will be able to provide the information. We will consider each request but may not be able to provide an alternative accessible format on every occasion.
## Reporting accessibility problems
We are working to meet accessibility standards across this website. If you experience difficulty in accessing our website please contact us at [SMLHelp@ons.gov.uk](mailto:SMLHelp@ons.gov.uk).
## Enforcement procedure
The Equality and Human Rights Commission (EHRC) is responsible for enforcing the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018 (the “accessibility regulations”). If you contact us with a complaint and you are not happy with our response contact the [Equality Advisory and Support Service (EASS)](https://www.equalityadvisoryservice.com/).
## Technical information about the website’s accessibility
The ONS is committed to making this portal accessible, in accordance with the [Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018](https://www.legislation.gov.uk/uksi/2018/952/made).
## What we’re doing to improve accessibility
We are committed to making sure our online services are accessible to all users and comply with level AA of the Web Content Accessibility Guidelines – WCAG 2.1 in line with the Public Sector Bodies (Websites and Mobile Applications) (No.2) Accessibility regulations 2018, and intend to.
In order to make sure our online services are accessible to all users, we are working with [The Digital Accessibility Centre (DAC)](http://www.digitalaccessibilitycentre.org/) to carry out WCAG 2.1 AA level technical compliance audits, which will also include testing by users with a wide range of disabilities.
As of November 2024, we are continuing this work to ensure that the Statistical Methods Library is compliant with the updated WCAG 2.2 guidelines to a AA level standard.
## Preparation of this accessibility statement
This statement was prepared on 1 February 2023, and last updated on 6 November 2024.— 06/11/2024 at 04:49:16 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 9
Updated At: 2024-11-07T14:16:54.654Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md— 07/11/2024 at 02:16:54 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-11-07T14:17:10.824Z
Content Updated:
id: selective-editing
name: Selective Editing
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: Selective Editing is an internationally recognised editing method where potential errors are prioritised according to their expected effect on key outputs. Only respondents that are having an impact on published estimates will be recontacted.
Selective Editing works by assigning a score to each important variable for a business where, the score reflects the impact that editing the respondent will have on the estimates. Only contributors with a high score are checked, low scoring contributors pass through unchecked.
To install this method, please see Help Centre - [Install a method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/selective-editing/selective_editing.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/selective_editing.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/selective_editing/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/selective-editing/selective_editing.md— 07/11/2024 at 02:17:10 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-11-07T14:17:47.529Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/thousand_pounds_correction/thousand_pounds_correction.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md— 07/11/2024 at 02:17:47 PM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-11-19T09:49:29.387Z
Content Updated:
id: metadata
term: Metadata
meaning: 'Data about data', giving information about a product itself, as opposed to its contents. In the case of the SML, metadata includes, for example: when a method was onboarded; its development status; by whom it was developed; whether or not it is approved for use in official outputs; etc.
api_name: glossary_entry/metadata— 19/11/2024 at 09:49:29 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-11-19T09:49:23.371Z
Content Updated:
id: pypi
term: PyPI
meaning: The Python Package Index (PyPI) is a repository of software for the Python programming language. Package authors (such as ONS-SML) use PyPI to distribute their software.
relatedlist: ["SML","Package","Repository"]
externalLinkTextShort: ONS-SML user profile on PyPI
externalUrl: https://pypi.org/user/ons-sml/
api_name: glossary_entry/pypi— 19/11/2024 at 09:49:23 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-11-19T09:49:26.353Z
Content Updated:
id: expert_group
term: Expert Group
meaning: A group of ONS employees within the Methodology and Quality Directorate with expertise in a specific area. For a full list, see https://analysisfunction.civilservice.gov.uk/government-statistical-service-and-statistician-group/gss-support/methodology/information-on-specific-methods/
externalLinkTextShort: Expert group
externalUrl: https://analysisfunction.civilservice.gov.uk/government-statistical-service-and-statistician-group/gss-support/methodology/information-on-specific-methods/
api_name: glossary_entry/expert_group— 19/11/2024 at 09:49:26 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-11-19T09:49:27.601Z
Content Updated:
id: release
term: Release
meaning: A 'snapshot' of a Git repository marked as being stable. Releases are commonly referred to by a version number. A popular scheme for determining version numbers is semantic versioning.
relatedlist: ["Git","Repository","Semantic Versioning"]
api_name: glossary_entry/release— 19/11/2024 at 09:49:27 AM —
Editor: Jason Bell
Environment: preprod
Content Type: methodsCatalogue
Revision: 1
Updated At: 2024-11-19T09:54:41.177Z
Content Updated: entry 78d7qRmxYSzox6ccAwoPII has been deleted.
heading1: {"en-US":"Methods catalogue"}
paragraph: {"en-US":"Here you will discover statistical methods used by the Office for National Statistics."}
detail: {"en-US":"What are method themes and expert groups?"}
detailContent: {"en-US":"Methods are sorted in hierarchical order with expert groups as the top level. For example, within an expert group are different themes, and within the themes are different methods.\n\nTo find out more about expert groups and themes, please visit the [analysis function website.](https://analysisfunction.civilservice.gov.uk/government-statistical-service-and-statistician-group/gss-support/methodology/information-on-specific-methods/)"}
api_name: {"en-US":"methods_catalogue_page/methods_catalogue"}— 19/11/2024 at 09:54:41 AM —
Editor: Jason Bell
Environment: preprod
Content Type: helpCentreInformation
Revision: 2
Updated At: 2024-11-19T09:56:10.371Z
Content Updated: entry 7ePz4NOKkNuYxTCKGceI0o has been deleted.
id: {"en-US":"get-in-touch"}
title: {"en-US":"Get in touch"}
content: {"en-US":"If you'd like to speak to someone on the SML team, please email [smlhelp@ons.gov.uk](mailto:smlhelp@ons.gov.uk).\n\nNeed help installing a method? Make sure you've consulted the Help Centre \"Install a Method\" page.\n\nNeed help using a method? Make sure you have read the user documentation, which you can find by clicking on the method name in the Method Catalogue. \n\nIf you still need assistance or if you have suggestions or other feedback, please do get in touch with us using the email above.\n"}
help_centre_category: {"en-US":"Feedback"}
api_name: {"en-US":"help_centre/get_in_touch"}— 19/11/2024 at 09:56:10 AM —
Editor: Jason Bell
Environment: preprod
Content Type: accessibilityPage
Revision: 1
Updated At: 2024-11-19T09:56:03.355Z
Content Updated:
title: Accessibility Statement
mainContent: # Accessibility Statement
This accessibility statement applies to the Office for National Statistics (ONS) Statistical Methods Library.
We want as many people as possible to be able to use the website. If you have a disability, [AbilityNet](https://mcmw.abilitynet.org.uk/) has advice on making your device easier to use.
## How accessible the website is
We are aware that some parts of this website are not fully accessible, and we are working to improve this.
## How to request content in an accessible format
If you need information from the website in a different format please contact us at [SMLHelp@ons.gov.uk](mailto:SMLHelp@ons.gov.uk) and tell us:
* The web address (URL).
* Your name and email address.
* The format you require.
We’ll reply within ten days to let you know when we will be able to provide the information. We will consider each request but may not be able to provide an alternative accessible format on every occasion.
## Reporting accessibility problems
We are working to meet accessibility standards across this website. If you experience difficulty in accessing our website please contact us at [SMLHelp@ons.gov.uk](mailto:SMLHelp@ons.gov.uk).
## Enforcement procedure
The Equality and Human Rights Commission (EHRC) is responsible for enforcing the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018 (the “accessibility regulations”). If you contact us with a complaint and you are not happy with our response contact the [Equality Advisory and Support Service (EASS)](https://www.equalityadvisoryservice.com/).
## Technical information about the website’s accessibility
The ONS is committed to making this portal accessible, in accordance with the [Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018](https://www.legislation.gov.uk/uksi/2018/952/made).
## What we’re doing to improve accessibility
We are committed to making sure our online services are accessible to all users and comply with level AA of the Web Content Accessibility Guidelines – WCAG 2.1 in line with the Public Sector Bodies (Websites and Mobile Applications) (No.2) Accessibility regulations 2018, and intend to.
In order to make sure our online services are accessible to all users, we are working with [The Digital Accessibility Centre (DAC)](http://www.digitalaccessibilitycentre.org/) to carry out WCAG 2.1 AA level technical compliance audits, which will also include testing by users with a wide range of disabilities.
As of November 2024, we are continuing this work to ensure that the Statistical Methods Library is compliant with the updated WCAG 2.2 guidelines to a AA level standard.
## Preparation of this accessibility statement
This statement was prepared on 1 February 2023, and last updated on 6 November 2024.
api_name: accessibility_page/accessibility_statement— 19/11/2024 at 09:56:03 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-11-19T09:56:03.375Z
Content Updated:
id: sml
term: SML
meaning: The Statistical Methods Library, a centralised collection of methods approved for use by the ONS. Comprises a code library, accessed through GitHub, and a collection of associated documentation, accessed via the SML Portal.
api_name: glossary_entry/sml— 19/11/2024 at 09:56:03 AM —
Editor: Jason Bell
Environment: preprod
Content Type: methodsCatalogue
Revision: 1
Updated At: 2024-11-19T09:56:14.613Z
Content Updated:
heading1: Methods catalogue
paragraph: Here you will discover statistical methods used by the Office for National Statistics.
detail: What are method themes and expert groups?
detailContent: Methods are sorted in hierarchical order with expert groups as the top level. For example, within an expert group are different themes, and within the themes are different methods.
To find out more about expert groups and themes, please visit the [analysis function website.](https://analysisfunction.civilservice.gov.uk/government-statistical-service-and-statistician-group/gss-support/methodology/information-on-specific-methods/)
api_name: methods_catalogue_page/methods_catalogue— 19/11/2024 at 09:56:14 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-11-19T09:56:12.345Z
Content Updated:
id: markdown
term: Markdown
meaning: A popular and simple markup language used across the Web for formatting text. Markdown is the format in which SML specifications are written.
api_name: glossary_entry/markdown— 19/11/2024 at 09:56:12 AM —
Editor: Jason Bell
Environment: preprod
Content Type: glossaryEntry
Revision: 1
Updated At: 2024-11-19T09:56:15.348Z
Content Updated:
id: git
term: Git
meaning: A popular software tool for version control. Files are stored in repositories (or 'repos'), which can be local or remote, and changes made are committed to a log, allowing files to be reverted to previous versions. A project can have multiple contributing authors and multiple 'branches' (variants) simultaneously, with Git providing tools to merge changes.
api_name: glossary_entry/git— 19/11/2024 at 09:56:15 AM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2024-11-27T09:11:47.962Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md
api_name: methods_table_and_summary/thousand_pound_correction— 27/11/2024 at 09:11:47 AM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2024-11-27T09:23:02.608Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/thousand_pounds_correction/thousand_pounds_correction.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md
api_name: methods_table_and_summary/thousand_pound_correction— 27/11/2024 at 09:23:02 AM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 4
Updated At: 2024-11-27T10:11:07.440Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/thousand_pounds_correction/thousand_pounds_correction.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/thousand-pound-correction/thousand_pound_correction.md
api_name: methods_table_and_summary/thousand_pound_correction— 27/11/2024 at 10:11:07 AM —
Editor: Joseph Percival
Environment: preprod
Content Type: accessibilityPage
Revision: 2
Updated At: 2024-12-04T16:46:26.530Z
Content Updated:
title: Accessibility Statement
mainContent: # Accessibility Statement
This accessibility statement applies to the Office for National Statistics (ONS) Statistical Methods Library.
We want as many people as possible to be able to use the website. If you have a disability, [AbilityNet](https://mcmw.abilitynet.org.uk/) has advice on making your device easier to use.
## How accessible the website is
We are aware that some parts of this website are not fully accessible, and we are working to improve this.
## How to request content in an accessible format
If you need information from the website in a different format please contact us at [SMLHelp@ons.gov.uk](mailto:SMLHelp@ons.gov.uk) and tell us:
* The web address (URL).
* Your name and email address.
* The format you require.
We’ll reply within ten days to let you know when we will be able to provide the information. We will consider each request but may not be able to provide an alternative accessible format on every occasion.
## Reporting accessibility problems
We are working to meet accessibility standards across this website. If you experience difficulty in accessing our website please contact us at [SMLHelp@ons.gov.uk](mailto:SMLHelp@ons.gov.uk).
## Enforcement procedure
The Equality and Human Rights Commission (EHRC) is responsible for enforcing the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018 (the “accessibility regulations”). If you contact us with a complaint and you are not happy with our response contact the [Equality Advisory and Support Service (EASS)](https://www.equalityadvisoryservice.com/).
## Technical information about the website’s accessibility
The ONS is committed to making this portal accessible, in accordance with the [Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018](https://www.legislation.gov.uk/uksi/2018/952/made).
## What we’re doing to improve accessibility
We are committed to making sure our online services are accessible to all users and comply with level AA of the Web Content Accessibility Guidelines – WCAG 2.2 in line with the Public Sector Bodies (Websites and Mobile Applications) (No.2) Accessibility regulations 2018.
In order to make sure our online services are accessible to all users, we are working with [The Digital Accessibility Centre (DAC)](http://www.digitalaccessibilitycentre.org/) to carry out WCAG 2.2 AA level technical compliance audits, which will also include testing by users with a wide range of disabilities.
## Preparation of this accessibility statement
This statement was prepared on 4 December 2024.
api_name: accessibility_page/accessibility_statement— 04/12/2024 at 04:46:26 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2025-01-22T11:13:24.706Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/DateAdjustment/DateAdjustment.md](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/DateAdjustment/DateAdjustment.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/DateAdjustment/DateAdjustment.md
api_name: methods_table_and_summary/date_adjustment— 22/01/2025 at 11:13:24 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2025-01-22T11:34:42.181Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/DateAdjustment/DateAdjustment.md)
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/DateAdjustment/DateAdjustment.md
api_name: methods_table_and_summary/date_adjustment— 22/01/2025 at 11:34:42 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 4
Updated At: 2025-01-22T11:44:25.626Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/DateAdjustment/DateAdjustment.md)
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md
api_name: methods_table_and_summary/date_adjustment— 22/01/2025 at 11:44:25 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 5
Updated At: 2025-01-22T11:50:47.951Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see User Notes (link below).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
codeLink: https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py
specificationLink: https://github.com/ONSdigital/Statistical-Method-Specifications/blob/main/editing_and_imputation/editing/date_adjustment/methodological_specification.md
userDocumentationLink: https://github.com/ONSdigital/sml-supporting-info/blob/main/method-info/date-adjustment/date_adjustment.md
api_name: methods_table_and_summary/date_adjustment— 22/01/2025 at 11:50:47 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2025-02-12T14:04:00.914Z
Content Updated:
id: selective-editing
name: Selective Editing
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: Selective Editing is an internationally recognised editing method where potential errors are prioritised according to their expected effect on key outputs. Only respondents that are having an impact on published estimates will be recontacted.
Selective Editing works by assigning a score to each important variable for a business where, the score reflects the impact that editing the respondent will have on the estimates. Only contributors with a high score are checked, low scoring contributors pass through unchecked.
To install this method, please see Help Centre - [Install a method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/SelectiveEditing/SelectiveEditing.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.3
urlList: ["[Go to Selective Editing code on GitHub](https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/selective_editing.py)","[Go to Selective Editing User Documentation on GitHub](https://github.com/ONSdigital/sml-user-doc)s/blob/main/UserGuides/SelectiveEditing/SelectiveEditing.md)"]
api_name: methods_table_and_summary/selective_editing— 12/02/2025 at 02:04:00 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 4
Updated At: 2025-02-12T14:47:58.387Z
Content Updated:
id: date-adjustment
name: Date Adjustment
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: To generate summary and headline statistics for a period, the ONS requires all businesses responses to cover the same period. However, sometimes it is not possible for a business to provide data for the exact period of days required. The responder may specify a different start and end dates for which the response totals cover. Therefore, Date Adjustment methods are required to approximate the values of the data for the required period that are based on the data received.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/DateAdjustment/DateAdjustment.md)
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.4
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
urlList: ["[Go to Date Adjustment User Documentation on GitHub](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/DateAdjustment/DateAdjustment.md)","[Go to Date Adjustment code on GitHub](https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/date_adjustment.py)"]
api_name: methods_table_and_summary/date_adjustment— 12/02/2025 at 02:47:58 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2025-02-12T14:57:57.653Z
Content Updated:
id: selective-editing
name: Selective Editing
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: Selective Editing is an internationally recognised editing method where potential errors are prioritised according to their expected effect on key outputs. Only respondents that are having an impact on published estimates will be recontacted.
Selective Editing works by assigning a score to each important variable for a business where, the score reflects the impact that editing the respondent will have on the estimates. Only contributors with a high score are checked, low scoring contributors pass through unchecked.
To install this method, please see Help Centre - [Install a method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/SelectiveEditing/SelectiveEditing.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.4
urlList: ["[Go to Selective Editing code on GitHub](https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/selective_editing.py)","[Go to Selective Editing User Documentation on GitHub](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/SelectiveEditing/SelectiveEditing.md)"]
api_name: methods_table_and_summary/selective_editing— 12/02/2025 at 02:57:57 PM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2025-02-13T08:27:17.707Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/ThousandPoundCorrection/ThousandPoundCorrection.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.4
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
urlList: ["[Go to Thousand Pound Correction code on GitHub](https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py)","[Go to Thousand Pound Correction User Documentation on GitHub](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/ThousandPoundCorrection/ThousandPoundCorrection.md)"]
api_name: methods_table_and_summary/thousand_pound_correction— 13/02/2025 at 08:27:17 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2025-02-13T08:41:14.836Z
Content Updated:
id: totals-and-components
name: Totals and Components
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for totals and components correction is currently used in ONS business surveys to ensure fixed relationships between variables are satisfied. For example, when a total (e.g., total employment) is collected along with the component breakdown (e.g., full-time male, full-time female, part-time male, part-time female). The primary use of the method is to automatically detect and correct errors in respondent data where fixed relationships have not been satisfied to improve the efficiency of the editing process, reduce the burden on respondents and survey validators and improve overall data quality. This method can also be used to ensure fixed relationships between variables are satisfied in other data types such as imputed data to improve overall data quality.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/TotalsAndComponents/TotalsAndComponents.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.4
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
urlList: ["[Go to Totals and Components code on GitHub](https://github.com/ONSdigital/sml-python-small/tree/main/sml_small/editing/totals_and_components/totals_and_components.py)","[Go to Totals and Components User Documentation on GitHub](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/TotalsAndComponents/TotalsAndComponents.md)"]
api_name: methods_table_and_summary/totals_and_components— 13/02/2025 at 08:41:14 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2025-02-13T08:52:17.708Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/CellKeyPerturbation/CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases
urlList: ["[Go to Cell Key Perturbation code on GitHub](https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py)","[Go to Cell Key Perturbation on GitHub](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/CellKeyPerturbation/CellKeyPerturbation_Py.md)"]
api_name: methods_table_and_summary/cell_key_perturbation— 13/02/2025 at 08:52:17 AM —
Editor: Sabrina Golonka
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2025-02-13T08:59:30.061Z
Content Updated:
id: cell-key-perturbation
name: Cell Key Perturbation
theme: Statistical Disclosure Control
expertGroup: Statistical Disclosure Control
language: Python
author: ONS
description: Cell-key Perturbation adds small amounts of noise to frequency tables, to protect against disclosure. Noise is added to change the counts that appear in the frequency table by small amounts, for example a 14 is changed to a 15. This noise introduces uncertainty in the counts and makes it harder to identify individuals, especially when taking the ‘difference’ between two similar tables. An input file called a ‘ptable’ is needed which specifies the level of perturbation.
To install and use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/CellKeyPerturbation/CellKeyPerturbation_Py.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v2.0.0
releaseLink: https://github.com/ONSdigital/cell-key-perturbation/releases
urlList: ["[Go to Cell Key Perturbation code on GitHub](https://github.com/ONSdigital/cell-key-perturbation/blob/main/cell_key_perturbation/create_perturbed_table.py)","[Go to Cell Key Perturbation User Documentation on GitHub](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/CellKeyPerturbation/CellKeyPerturbation_Py.md)"]
api_name: methods_table_and_summary/cell_key_perturbation— 13/02/2025 at 08:59:30 AM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 2
Updated At: 2025-03-12T13:48:36.040Z
Content Updated:
id: thousand-pound-correction test
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/ThousandPoundCorrection/ThousandPoundCorrection.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.4
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
urlList: ["[Go to Thousand Pound Correction code on GitHub](https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py)","[Go to Thousand Pound Correction User Documentation on GitHub](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/ThousandPoundCorrection/ThousandPoundCorrection.md)"]
api_name: methods_table_and_summary/thousand_pound_correction— 12/03/2025 at 01:48:36 PM —
Editor: Jason Bell
Environment: preprod
Content Type: catalogueTableOfMethods2
Revision: 3
Updated At: 2025-03-12T13:49:37.332Z
Content Updated:
id: thousand-pound-correction
name: Thousand Pound Correction
theme: Editing
expertGroup: Editing & Imputation
language: Python
author: ONS
description: The automatic editing method for thousand pounds correction is commonly used across ONS business surveys. It is a generic rule that uses user defined thresholds to automatically detect and correct thousand pounds errors. This is when the respondent should have reported values in thousands of pounds but has reported in actual pounds e.g., returned a value of £56,000 instead of correctly submitting 56.
The method checks the ratio of a principal variable against a suitable predictor variable and make an automatic correction if the calculated ratio is around 1000. The automatic correction is applied to all relevant variables for the respondent.
To install this method, please see Help Centre - [Install a Method](https://statisticalmethodslibrary.ons.gov.uk/help-centre/access/install-a-method).
To use this method, please see [User Notes](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/ThousandPoundCorrection/ThousandPoundCorrection.md).
contactDetails: smlhelp@ons.gov.uk
methodReady: true
releaseVersion: v1.2.4
releaseLink: https://github.com/ONSdigital/sml-python-small/releases
urlList: ["[Go to Thousand Pound Correction code on GitHub](https://github.com/ONSdigital/sml-python-small/blob/main/sml_small/editing/thousand_pounds/thousand_pounds.py)","[Go to Thousand Pound Correction User Documentation on GitHub](https://github.com/ONSdigital/sml-user-docs/blob/main/UserGuides/ThousandPoundCorrection/ThousandPoundCorrection.md)"]
api_name: methods_table_and_summary/thousand_pound_correction— 12/03/2025 at 01:49:37 PM —