diff --git a/DESCRIPTION b/DESCRIPTION index 924dc07f..128ee129 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -17,7 +17,10 @@ Authors@R: c( person('Nicholas', 'Lee', role = c('ctb')), person('Ruben', 'Jacobo', role = c('ctb')), person('Waylon', 'Ho', role = c('ctb')), - person('Nicole', 'Hoess', role = c('ctb')) + person('Nicole', 'Hoess', role = c('ctb')), + person('Anthony', 'Lau', role = c('ctb')), + person('Sean', 'Sunoo', role = c('ctb')), + person('Ian Jaymes', 'Iwata', role= c('ctb')) ) Maintainer: Carlos Paradis License: MPL-2.0 | file LICENSE diff --git a/NAMESPACE b/NAMESPACE index ebf58489..9f95bc54 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -12,6 +12,9 @@ export(download_bugzilla_perceval_traditional_issue_comments) export(download_bugzilla_rest_comments) export(download_bugzilla_rest_issues) export(download_bugzilla_rest_issues_comments) +export(download_jira_issues) +export(download_jira_issues_by_date) +export(download_jira_issues_by_issue_key) export(download_mod_mbox) export(download_mod_mbox_per_month) export(download_pipermail) @@ -121,6 +124,7 @@ export(parse_gitlog_entity) export(parse_gof_patterns) export(parse_java_code_refactoring_json) export(parse_jira) +export(parse_jira_latest_date) export(parse_jira_replies) export(parse_jira_rss_xml) export(parse_line_metrics) @@ -136,6 +140,7 @@ export(query_src_text_class_names) export(query_src_text_namespace) export(read_temporary_file) export(recolor_network_by_community) +export(refresh_jira_issues) export(smell_missing_links) export(smell_organizational_silo) export(smell_radio_silence) diff --git a/NEWS.md b/NEWS.md index e74bf146..d8536297 100644 --- a/NEWS.md +++ b/NEWS.md @@ -3,6 +3,11 @@ __kaiaulu 0.0.0.9700 (in development)__ ### NEW FEATURES * `github_api_project_issue_refresh` and `github_api_project_issue_or_pr_comment_refresh` were added to download issue data or comments respectively that have not already been downloaded. + * `refresh_jira_issues()` had been added. It is a wrapper function for the previous downloader and downloads only issues greater than the greatest key already downloaded. + * `download_jira_issues()`, `download_jira_issues_by_issue_key()`, and `download_jira_issues_by_date()` has been added. This allows for downloading of Jira issues without the use of JirAgileR [#275](https://github.com/sailuh/kaiaulu/issues/275) and specification of issue Id and created ranges. It also interacts with `parse_jira_latest_date` to implement a refresh capability. + * `make_jira_issue()` and `make_jira_issue_tracker()` no longer create fake issues following JirAgileR format, but instead the raw data obtained from JIRA API. This is compatible with the new parser function for JIRA. [#277](https://github.com/sailuh/kaiaulu/issues/277) + * `parse_jira()` now parses folders containing raw JIRA JSON files without depending on JirAgileR. [#276](https://github.com/sailuh/kaiaulu/issues/276) + * The `parse_jira_latest_date()` has been added. This function returns the file name of the downloaded JIRA JSON containing the latest date for use by `download_jira_issues()` to implement a refresh capability. [#276](https://github.com/sailuh/kaiaulu/issues/276) * Kaiaulu architecture has been refactored. Instead of using a parser, download, network module structure, Kaiaulu now uses a combination of data type and tool structure. In that manner, various parser functions of download,R, parser.R, and network.R now are separated in git.R, jira.R, git.R, etc. When only small functionality of a tool is required, functions are grouped based on the data type they are associated to, for example, src.R. Kaiaulu API documentation has been updated accordingly. Functions signature and behavior remain the same: The only modification was the new placement of functions into files. For further rationale and changes, see the issue for more details. [#241](https://github.com/sailuh/kaiaulu/issues/241) * Temporal bipartite projections are now weighted. The temporal projection can be parameterized by `weight_scheme_cum_temporal()` `weight_scheme_pairwise_cum_temporal()` when all time lag edges are used, or the existing weight schemes can also be used when using a single lag. The all lag weight schemes reproduce the same behavior as Codeface's paper. See the issue for details. [#229](https://github.com/sailuh/kaiaulu/issues/229) * The `make_jira_issue()` and `make_jira_issue_tracker()` have been added, alongside examples and unit tests for `parse_jira()`. [#228](https://github.com/sailuh/kaiaulu/issues/228) diff --git a/R/example.R b/R/example.R index 757cec4a..f70bd5c6 100644 --- a/R/example.R +++ b/R/example.R @@ -396,34 +396,39 @@ example_notebook_alternating_function_in_files <- function(folder_path="/tmp", f #' #' @param folder_path The path where the folder will be created #' @param folder_name The name of the folder -#' @return the JSON path of the newly created issue issue tracker +#' @return the JSON folder path of the newly created issue issue tracker #' @export #' @keywords internal -example_jira_issue_components <- function(folder_path="/tmp",folder_name) { +example_jira_issue_components <- function(folder_path = "/tmp", folder_name) { # Create folder & repo - folder_path <- io_make_folder(folder_path=folder_path, folder_name = folder_name) - - issue1 <- make_jira_issue(jira_domain_url = "https://project.org/jira", - issue_key = "GERONIMO-123", - issue_type = "New Feature", - status = "Open", - resolution = "Finished", - title = "This is a summary.", - description = "The new features have been implemented.", - components = "x-core;x-spring", - creator_name = "Bob", - reporter_name = "Joe", - assignee_name = "Moe", + folder_path <- io_make_folder(folder_path = folder_path, folder_name = folder_name) + + issue1 <- make_jira_issue( + jira_domain_url = "https://project.org/jira", + issue_key = "PROJECT-123", + project_key = "PROJECT", + summary = "Summary of new feature", + description = "The new features have been implemented", + issue_type = "New Feature", + resolution = "Finished", + priority = "Minor", + status = "Open", + labels = c("pull-request-available"), + components = c("jira", "mail"), + affects_versions = c("3.4.3"), + fix_versions = c("3.4.2"), + assignee_name = "Moe", + creator_name = "Bob", + reporter_name = "Joe" ) - # issues <- c(list(issue1)) issues <- list(issue1) jira_json_path <- make_jira_issue_tracker(issues, - save_filepath=file.path(folder_path,"issue_two_components.json")) + save_filepath=file.path(folder_path, "ONE_ISSUE_NO_COMMENTS_issues_1121646814_1121719175.json")) - return(jira_json_path) + return(folder_path) } #' Example JIRA Issue Tracker No Comments @@ -432,47 +437,58 @@ example_jira_issue_components <- function(folder_path="/tmp",folder_name) { #' #' @param folder_path The path where the folder will be created #' @param folder_name The name of the folder -#' @return the JSON path of the newly created issue issue tracker +#' @return the JSON folder path of the newly created issue issue tracker #' @export #' @keywords internal -example_jira_two_issues <- function(folder_path="/tmp",folder_name) { +example_jira_two_issues <- function(folder_path = "/tmp", folder_name) { # Create folder & repo - folder_path <- io_make_folder(folder_path=folder_path, folder_name = folder_name) - - issue1 <- make_jira_issue(jira_domain_url = "https://project.org/jira", - issue_key = "GERONIMO-123", - issue_type = "New Feature", - status = "Open", - resolution = "Finished", - title = "This is a summary.", - description = "The new features have been implemented.", - components = "x-core", - creator_name = "Bob", - reporter_name = "Joe", - assignee_name = "Moe", + folder_path <- io_make_folder(folder_path = folder_path, folder_name = folder_name) + + issue1 <- make_jira_issue( + jira_domain_url = "https://project.org/jira", + issue_key = "PROJECT-11", + project_key = "PROJECT", + summary = "Summary of issue 1", + description = "Description of summary 1", + issue_type = "New Feature", + resolution = "Finished", + priority = "Minor", + status = "Closed", + labels = c("pull-request-available"), + components = c("jira"), + affects_versions = c("1.1.1"), + fix_versions = c("1.1.1"), + assignee_name = "Moe", + creator_name = "Bob", + reporter_name = "Joe" ) - issue2 <- make_jira_issue(jira_domain_url = "https://project.org/jira", - issue_key = "GERONIMO-124", - issue_type = "New Feature", - status = "Open", - resolution = "Finished", - title = "This is a summary.", - description = "The new features have been implemented.", - components = "x-spring", - creator_name = "Moe", - reporter_name = "Larry", - assignee_name = "Curly", + issue2 <- make_jira_issue( + jira_domain_url = "https://project.org/jira", + issue_key = "PROJECT-22", + project_key = "PROJECT", + summary = "Summary of issue 2", + description = "Description of summary 2", + issue_type = "New Feature", + resolution = "Finished", + priority = "Minor", + status = "Open", + labels = c("pull-request-available"), + components = c("jira"), + affects_versions = c("2.2.2"), + fix_versions = c("2.2.2"), + assignee_name = "Steven", + creator_name = "Nathan", + reporter_name = "Matthew" ) - # issues <- c(list(issue1), list(issue2)) issues <- list(issue1, issue2) jira_json_path <- make_jira_issue_tracker(issues, - save_filepath=file.path(folder_path,"issue_with_components.json")) + save_filepath=file.path(folder_path, "TWO_ISSUES_NO_COMMENTS_issues_1121646814_1121719175.json")) - return(jira_json_path) + return(folder_path) } #' Example Jira Issue Tracker With Comments @@ -481,38 +497,43 @@ example_jira_two_issues <- function(folder_path="/tmp",folder_name) { #' #' @param folder_path The path where the folder will be created #' @param folder_name The name of the folder -#' @return the JSON path of the newly created issue issue tracker +#' @return the JSON folder path of the newly created issue issue tracker #' @export #' @keywords internal -example_jira_issue_comments <- function(folder_path="/tmp",folder_name) { +example_jira_issue_comments <- function(folder_path = "/tmp", folder_name) { # Create folder & repo - folder_path <- io_make_folder(folder_path=folder_path, folder_name = folder_name) - - issue1 <- make_jira_issue(jira_domain_url = "https://project.org/jira", - issue_key = "GERONIMO-2", - issue_type = "Bug", - status = "In Progress", - resolution = "Incomplete", - title = "Bug fixes need implementation.", - description = "The new features have not been implemented.", - components = "x-core", - creator_name = "Moe", - reporter_name = "Larry", - assignee_name = "Curly", - comments = c( - "This is the first body comment.", - "This is the second body comment." - ) + folder_path <- io_make_folder(folder_path = folder_path, folder_name = folder_name) + + issue1 <- make_jira_issue( + jira_domain_url = "https://project.org/jira", + issue_key = "PROJECT-123", + project_key = "PROJECT", + summary = "Summary of new feature", + description = "The new features have been implemented", + issue_type = "New Feature", + resolution = "Finished", + priority = "Minor", + status = "Open", + labels = c("pull-request-available"), + components = c("jira", "mail"), + affects_versions = c("3.4.3"), + fix_versions = c("3.4.2"), + assignee_name = "Moe", + creator_name = "Bob", + reporter_name = "Joe", + comments = c( + "This is the first body comment.", + "This is the second body comment." + ) ) - # issues <- c(list(issue1)) issues <- list(issue1) - jira_json_path <- make_jira_issue_tracker(issues, - save_filepath=file.path(folder_path,"one_issue_two_comments.json")) + jira_json_path <- make_jira_issue_tracker( + issues, save_filepath=file.path(folder_path,"ONE_ISSUE_WITH_COMMENTS_issues_1121646814_1121719175.json")) - return(jira_json_path) + return(folder_path) } #' Two Thread and Three Replies Mailing List diff --git a/R/jira.R b/R/jira.R index 1c51e745..5ed88e16 100644 --- a/R/jira.R +++ b/R/jira.R @@ -4,18 +4,533 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this # file, You can obtain one at https://mozilla.org/MPL/2.0/. +############## Downloaders ############## + +#' Download JIRA Issues and/or Comments +#' +#' Download JIRA issues and/or comments using [rest/api/2/search](https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issue-search/#api-rest-api-2-search-post) endpoint +#' to the specified `save_folder_path`. +#' +#' The folder assumes the following convention: "(PROJECTKEY)_(uniextimestamp_lowerbound)_(unixtimestamp_upperbound).json" +#' For example: "SAILUH_1231234_2312413.json". +#' +#' Comments data are added when the field `comment` is included. +#' +#' If a project requires authentication and authentication fails, the function will end without downloading any data. +#' +#' If the number of results per page returned is less than the number specified, the max_results value will adjust to that value. +#' +#' @param domain Custom JIRA domain URL +#' @param username the JIRA username +#' @param password the JIRA password/API token +#' @param jql_query Specific query string to specify criteria for fetching +#' @param fields List of fields that are downloaded in each issue +#' @param save_folder_path Path that files will be saved in .json format +#' @param max_results (optional) the [maximum number of results](https://confluence.atlassian.com/jirakb/how-to-use-the-maxresults-api-parameter-for-jira-issue-search-rest-api-1251999998.html) +#' to download per page. Default is 50. +#' @param max_total_downloads Maximum downloads per function call. +#' This parameter specifies how many issues should be downloaded. Subsequent API calls will stop if they +#' reach or surpass this value. +#' @param search_query an optional API parameter that alters the GET request. +#' See \code{\link{download_jira_issues_by_date}} and \code{\link{download_jira_issues_by_issue_key}} +#' source code for examples. +#' @param verbose Set verbose=TRUE to print execution details. +#' @export +#' @family jira +#' @family downloaders +#' @seealso \code{\link{download_jira_issues_by_date}} to download JIRA issues and/or comments by date range +#' @seealso \code{\link{download_jira_issues_by_issue_key}} to download JIRA issues and/or comments by issue key range +#' @seealso \code{\link{parse_jira}} to parse jira data with or without comments +#' @seealso \code{\link{refresh_jira_issues}} to obtain more recent data from any of the downloader functions +download_jira_issues <- function(domain, + jql_query, + fields, + username = NULL, + password = NULL, + save_folder_path, + max_results = 50, + max_total_downloads = 5000, + search_query = NULL, + verbose = FALSE) { + + # Ensure the domain starts with https:// for secure communication. + if (!grepl("^https?://", domain)) { + domain <- paste0("https://", domain) + } + + # append search_query to jql_query if present + if (!is.null(search_query)){ + jql_query <- paste(jql_query, search_query) + if(verbose){ + message(jql_query) + } + } + + # Initialize variables for pagination + start_at <- 0 + total <- max_results + all_issues <- list() + # This variable counts your download count. This is important to not exceed the max downloads per hour + download_count <- 0 + time <- Sys.time() + if(verbose){ + message("Starting Downloads at ", time) + } + # Loop that downloads each issue into a file + repeat{ + + # Check update our download count and see if it is approaching the limit + if (download_count + max_results > max_total_downloads) { + # error message + time <- Sys.time() + if(verbose){ + message("Cannot download as max_total_downloads will be exceeded. Try again at a later time. Download ended at ", time) + } + break + } # Resume downloading + + # Construct the API endpoint URL + url <- httr::parse_url(domain) + + #Authenticate if username and password are provided + if(!is.null(username)&&!is.null(password)) { + # Use the username/password for authentication + auth <- httr::authenticate(as.character(username), as.character(password), "basic") + #message("successfully authenticated") + } else { + if(verbose){ + message("No username or password present or are formatted incorrectly.") + } + auth <- NULL + } + + url<-httr::modify_url(url = url, + scheme = if(is.null(url$scheme)){"https"}, + path = c(url$path,"/rest/api/2/search"), + query=list(jql = jql_query, + fields = paste(fields, collapse = ","), + startAt = start_at, + maxResults = max_results)) + + url <- httr::parse_url(url) + url_built <- httr::build_url(url) + + # Make the API call + if(!is.null(username)&&!is.null(password)){ + response <- httr::GET(url_built, + if(verbose){httr::verbose()}, + auth, + httr::user_agent("github.com/sailuh/kaiaulu")) + }else{ + response <- httr::GET(url_built, + if(verbose){httr::verbose()}, + httr::user_agent("github.com/sailuh/kaiaulu")) + } + + if(verbose){ + message("Requested: ", response$request$url) + } + + # Stop if there's an HTTP error + if (httr::http_error(response)) { + stop("API request failed: ", httr::http_status(response)$message) + } + + # Extract issues. for iteration of naming convention and checks + r_object_content <- jsonlite::fromJSON(httr::content(response, "text", encoding = "UTF-8"), + simplifyVector = FALSE) + # The number of issues downloaded + issue_count <- length(r_object_content$issues) + # save the raw content for a writeLines later + raw_content <- httr::content(response, "text", encoding = "UTF-8") + + # Check to make sure that the api is downloading the correct amount of issues specified by max_results + # This checks for only the first page (if download_count ==0) + # If the total number of issues retrieved is less than max_results, then of course issue_count + # will be < maxResults so we check to make sure this is not true (total >= max_results) + if ((download_count == 0) && (max_results != issue_count)) { + if(verbose){ + message(". max_results specified: ", max_results) + message(". Number of issues retrieved: ", issue_count) + message(". Something went wrong with the API request. Changing max_results to ", issue_count) + } + max_results <- issue_count + } + + if (issue_count > 0){ + + # Construct JSON file name based on refresh convention + + + # The file name prefix is the JIRA project key. The from and to unix timestamps are then added. + file_name <- stringi::stri_extract_first_regex(str=jql_query, pattern="(?<=project=')[:alpha:]+") + + # Extract 'created' dates + created_dates <- sapply(r_object_content$issues, function(issue) issue$fields$created) + + # Convert to POSIXct date objects + date_objects <- as.POSIXct(created_dates, format="%Y-%m-%dT%H:%M:%S", tz="UTC") + + # Find the greatest and smallest date + latest_date <- max(date_objects) + latest_date_unix <- as.numeric(latest_date) + oldest_date <- min(date_objects) + oldest_date_unix <- as.numeric(oldest_date) + + # Append oldest and latest dates to the file name + file_name <- paste0(file_name, "_", oldest_date_unix) + file_name <- paste0(file_name, "_", latest_date_unix, ".json") + + + + # Add path where file will be saved + file_name <- file.path(save_folder_path,file_name) + + # Print the latest and oldest dates and file name + if (verbose){ + message("Latest date:", latest_date_unix) + message("Oldest date:", oldest_date_unix) + message("File name: ", file_name) + } + } + + # write the files if issues present + if (issue_count > 0){ + writeLines(raw_content, file_name) + } else { + if(verbose){ + message("You are all caught up!") + } + } + + # update download_count and optional print statements + download_count <- download_count + issue_count + if (verbose && (issue_count > 0)){ + message("saved file to ", file_name) + message("Saved ", download_count, " total issues") + } + + + #updates start_at for next loop + if (issue_count < max_results) { + break + } else { + start_at <- start_at + max_results + } + } + + # Final verbose output + if (verbose) { + message("Success! Fetched and saved issues.") + } + + # Returns the content so that it can be saved to a variable via function call + return(NULL) +} + + +#' Download JIRA Issues and/or Comments by Date Range +#' +#' Wraps around \code{\link{download_jira_issues}} providing JQL query parameters for specifying date ranges. +#' Only issues created in the specified date range will be downloaded. +#' +#' Acceptable formats for `date_lower_bound` and `date_upper_bound` are: +#' +#' * "yyyy/MM/dd HH:mm" +#' * "yyyy-MM-dd HH:mm" +#' * "yyyy/MM/dd" +#' * "yyyy-MM-dd" +#' +#' For example: `date_lower_bound=2023/11/16 21:00` (an issue ocurring at the exact specified time will also be downloaded). +#' +#' For further details on the `created` JQL Query see [the associated JIRA API documentation](https://support.atlassian.com/jira-software-cloud/docs/jql-fields/#Created). +#' +#' @param domain Custom JIRA domain URL +#' @param username the JIRA username +#' @param password the JIRA password/API token +#' @param jql_query Specific query string to specify criteria for fetching +#' @param fields List of fields that are downloaded in each issue +#' @param save_folder_path Path that files will be saved in .json format +#' @param max_results (optional) the [maximum number of results](https://confluence.atlassian.com/jirakb/how-to-use-the-maxresults-api-parameter-for-jira-issue-search-rest-api-1251999998.html) +#' to download per page. Default is 50. +#' @param max_total_downloads Maximum downloads per function call. +#' This parameter specifies how many issues should be downloaded. Subsequent API calls will stop if they +#' reach or surpass this value. +#' See \code{\link{download_jira_issues_by_date}} and \code{\link{download_jira_issues_by_issue_key}} +#' source code for examples. +#' @param date_lower_bound Optional. Specify the lower bound date time (e.g. 2023/11/16 21:00) +#' @param date_upper_bound Optional. Specify the upper bound date time (e.g. 2023/11/17 21:00) +#' @param verbose Set verbose=TRUE to print execution details. +#' @export +#' @family jira +#' @family downloaders +#' @seealso \code{\link{download_jira_issues_by_issue_key}} to download JIRA issues and/or comments by issue key range +#' @seealso \code{\link{download_jira_issues}} for more flexibility in specifying the JQL query +#' @seealso \code{\link{parse_jira}} to parse jira data with or without comments +#' @seealso \code{\link{refresh_jira_issues}} to obtain more recent data from any of the downloader functions +download_jira_issues_by_date <- function(domain, + jql_query, + fields, + username = NULL, + password = NULL, + save_folder_path, + max_results, + max_total_downloads, + date_lower_bound = NULL, + date_upper_bound = NULL, + verbose){ + created_query <- "" + if (!is.null(date_lower_bound)){ + created_query <- paste0(created_query, "AND created >= '", date_lower_bound, "' ") + } + if (!is.null(date_upper_bound)){ + created_query <- paste0(created_query, "AND created <= '", date_upper_bound, "' ") + } + if(verbose){ + message("Appending ", created_query, " to api request.") + } + + download_jira_issues(domain = domain, + jql_query = jql_query, + fields = fields, + username = username, + password = password, + save_folder_path = save_folder_path, + max_results = max_results, + max_total_downloads = max_total_downloads, + search_query = created_query, + verbose = verbose) +} + +#' Download JIRA Issues and/or Comments by Issue Key Range +#' +#' #' Wraps around \code{\link{download_jira_issues}} providing jql query parameters for specifying issue key ranges. +#' Only issues created in the specified date range inclusive will be downloaded. +#' +#' The acceptable format for `issue_key_lower_bound` and `issue_key_upper_bound` is: - +#' +#' For example: `issue_key_lower_bound=SAILUH-1` (SAILUH-1 will also be downloaded). +#' +#' For further details on the `issueKey` JQL Query see [the associated JIRA API documentation](https://support.atlassian.com/jira-software-cloud/docs/jql-fields/#Issue-key) +#' +#' +#' @param domain Custom JIRA domain URL +#' @param username the JIRA username +#' @param password the JIRA password/API token +#' @param jql_query Specific query string to specify criteria for fetching +#' @param fields List of fields that are downloaded in each issue +#' @param save_folder_path Path that files will be saved in .json format +#' @param max_results (optional) the [maximum number of results](https://confluence.atlassian.com/jirakb/how-to-use-the-maxresults-api-parameter-for-jira-issue-search-rest-api-1251999998.html) +#' to download per page. Default is 50. +#' @param max_total_downloads Maximum downloads per function call. +#' This parameter specifies how many issues should be downloaded. Subsequent API calls will stop if they +#' reach or surpass this value. +#' See \code{\link{download_jira_issues_by_date}} and \code{\link{download_jira_issues_by_issue_key}} +#' source code for examples. +#' @param issue_key_lower_bound Optional. Specify the lower bound issue key (e.g. SAILUH-1) +#' @param issue_key_upper_bound Optional. Specify the upper bound issue key (e.g. SAILUH-3) +#' @param verbose Set verbose=TRUE to print execution details. +#' @export +#' @family jira +#' @family downloaders +#' @seealso \code{\link{download_jira_issues_by_date}} to download JIRA issues and/or comments by date range +#' @seealso \code{\link{download_jira_issues}} for more flexibility in specifying the JQL query +#' @seealso \code{\link{parse_jira}} to parse jira data with or without comments +#' @seealso \code{\link{refresh_jira_issues}} to obtain more recent data from any of the downloader functions +download_jira_issues_by_issue_key <- function(domain, + jql_query, + fields, + username = NULL, + password = NULL, + save_folder_path, + max_results, + max_total_downloads, + issue_key_lower_bound = NULL, + issue_key_upper_bound = NULL, + verbose){ + created_query <- "" + if (!is.null(issue_key_lower_bound)){ + created_query <- paste0(created_query, "AND issueKey >= ", issue_key_lower_bound) + } + if (!is.null(issue_key_upper_bound)){ + created_query <- paste0(created_query, " AND issueKey <= ", issue_key_upper_bound) + } + if(verbose){ + message("Appending ", created_query, " to api request.") + } + + download_jira_issues(domain = domain, + jql_query = jql_query, + fields = fields, + username = username, + password = password, + save_folder_path = save_folder_path, + max_results = max_results, + max_total_downloads = max_total_downloads, + search_query = created_query, + verbose) +} + +#' Refresh JIRA Issues and/or Comments +#' +#' Uses the adopted file name convention by \code{\link{download_jira_issues}} to identify +#' the latest downloaded JIRA issue key KEY-i, and calls +#' \code{\link{download_jira_issues_by_issue_key}} with lower bound KEY-(i+1) to download all +#' newer issues. +#' +#' If the directory is empty, then all issues will be downloaded. This function can therefore +#' be used in the specified folder to continuously refresh available issues and/or comments +#' data. +#' + +#' @param domain Custom JIRA domain URL +#' @param username the JIRA username +#' @param password the JIRA password/API token +#' @param jql_query Specific query string to specify criteria for fetching +#' @param fields List of fields that are downloaded in each issue +#' @param save_folder_path Path that files will be saved in .json format +#' @param max_results (optional) the [maximum number of results](https://confluence.atlassian.com/jirakb/how-to-use-the-maxresults-api-parameter-for-jira-issue-search-rest-api-1251999998.html) +#' to download per page. Default is 50. +#' @param max_total_downloads Maximum downloads per function call. +#' This parameter specifies how many issues should be downloaded. Subsequent API calls will stop if they +#' reach or surpass this value. +#' @param verbose Set verbose=TRUE to print execution details. +#' @export +#' @family downloaders +#' @family jira +#' @seealso \code{\link{download_jira_issues_by_date}} to download JIRA issues and/or comments by date range +#' @seealso \code{\link{download_jira_issues_by_issue_key}} to download JIRA issues and/or comments by issue key range +#' @seealso \code{\link{download_jira_issues}} for more flexibility in specifying the JQL query +#' @seealso \code{\link{parse_jira}} to parse jira data with or without comments +#' @seealso \code{\link{parse_jira_latest_date}} to retrieve the file path of the latest issue key +refresh_jira_issues <- function(domain, + jql_query, + fields, + username = NULL, + password = NULL, + save_folder_path, + max_results, + max_total_downloads, + verbose){ + + # List all files and subdirectories in the directory + existing_issues <- list.files(path = save_folder_path) + + # If the folder is empty, then start by downloading all issues. + if(length(existing_issues) == 0) { + if(verbose){ + message("The folder is empty. Downloading all issues. \n") + } + download_jira_issues(domain = domain, + jql_query = jql_query, + fields = fields, + username = username, + password = password, + save_folder_path = save_folder_path, + max_results = max_results, + max_total_downloads = max_total_downloads, + search_query = NULL, + verbose = verbose) + } else { + # If folder is not empty, find the highest issue key, and resume download after it. + # First, get the file name with the last downloaded 'issueKey' value + file_name_with_greatest_issue_key <- parse_jira_latest_date(save_folder_path) + + # Prepare the path and filename + issue_refresh <- file.path(save_folder_path, file_name_with_greatest_issue_key) + + # Check if the file exists + if(file.exists(issue_refresh)) { + # Check if the file is empty by checking its size + } else { + if(verbose){ + stop("The file does not exist.\n") + } + } + if(verbose){ + message("Filename with highest date (and therefore latest issue key is inside): ", issue_refresh) + } + + # Read the JSON file + json_data <- jsonlite::fromJSON(txt = issue_refresh, simplifyVector = FALSE) + + # Extract the Maximum issue key value + # Start with a low value assuming no negative numbers + max_numeric_part <- -1 + max_key <- "" + + for (issue in json_data$issues) { + # Extract the key for the current issue + current_key <- issue$key + + # Extract the numeric part of the key + # Assuming the key format is "PROJECTNAME-NUMBER" + numeric_part <- as.numeric(sub("^[^-]+-", "", current_key)) + + # Check if the numeric part is greater than the current maximum + if (numeric_part > max_numeric_part) { + # Update the maximum numeric part and the corresponding key + max_numeric_part <- numeric_part + max_key <- current_key + } + } + + # Print the key with the maximum numeric part + if(verbose){ + message("The greatest issue key value is ", max_key) + } + + # Construct the search query to append to the JIRA API request + search_query <- paste0("AND issueKey > ", max_key) + if(verbose){ + message("Appending ", search_query, " to JQL query") + } + + # Call the downloader with appended query + download_jira_issues(domain = domain, + jql_query = jql_query, + fields = fields, + username = username, + password = password, + save_folder_path = save_folder_path, + max_results = max_results, + max_total_downloads = max_total_downloads, + search_query = search_query, + verbose = verbose) + } +} + + ############## Parsers ############## -#' Parse Jira issue and comments +#' Parse JIRA Issues and Comments +#' +#' Parses JIRA issues without or with comments contained in a folder following a standardized file nomenclature. +#' as obtained from \code{\link{download_jira_issues}}. A named list with two elements (issues, comments) is returned +#' containing the issue table and optionally comments table. #' -#' @param json_path path to jira json (issues or issues with comments) obtained using `download_jira_data.Rmd`. +#' The following fields are expected on the raw data: +#' +#' issuekey, issuetype, components, creator, created, description, reporter, status, resolution +#' resolutiondate, assignee, updated, comment, priority, votes, watches, versions, fixVersions, labels +#' +#' which are the default parameters of \code{\link{download_jira_issues}}. If the `comment` field is +#' specified, then the comments table is included. +#' +#' If a field is not present in an issue, then its value will be NA. +#' +#' +#' @param json_folder_path is a folder path containing a set of jira_issues as json files. #' @return A named list of two named elements ("issues", and "comments"), each containing a data.table. -#' Note the comments element will be empty if the downloaded json only contain issues. #' @export #' @family parsers -parse_jira <- function(json_path){ +parse_jira <- function(json_folder_path){ - json_issue_comments <- jsonlite::read_json(json_path) + file_list <- list.files(json_folder_path) + + if (identical(file_list, character(0))){ + stop(stringi::stri_c("cannot open the connection")) + } # Comments list parser. Comments may occur on any json issue. jira_parse_comment <- function(comment){ @@ -38,78 +553,141 @@ parse_jira <- function(json_path){ return(parsed_comment) } - # names(json_issue_comments) => "base_info","ext_info" - # length([["base_info]]) == length([["ext_info]]) == n_issues. - # Choose either and store the total number of issues - n_issues <- length(json_issue_comments[["ext_info"]]) + # Issues parser + jira_parse_issues <- function(jira_file){ + + json_issue_comments <- jsonlite::read_json(jira_file) + + n_issues <- length(json_issue_comments[["issues"]]) + + # Prepare two lists which will contain data.tables for all issues and all comments + # Both tables can share the issue_key, so they can be joined if desired. + all_issues <- list() + all_issues_comments <- list() + + for(i in 1:n_issues){ + + # This is the issue key + issue_key <- json_issue_comments[["issues"]][[i]][["key"]][[1]] + + # All other information is contained in "fields" + issue_comment <- json_issue_comments[["issues"]][[i]][["fields"]] + + # Parse all relevant *issue* fields + all_issues[[i]] <- data.table( + issue_key = issue_key, + + issue_summary = issue_comment[["summary"]][[1]], + issue_parent = issue_comment[["parent"]][["name"]][[1]], + issue_type = issue_comment[["issuetype"]][["name"]][[1]], + issue_status = issue_comment[["status"]][["statusCategory"]][["name"]][[1]], + issue_resolution = issue_comment[["resolution"]][["name"]][[1]], + issue_components = stringi::stri_c(unlist(sapply(issue_comment[["components"]],"[[","name")),collapse = ";"), + issue_description = issue_comment[["description"]][[1]], + issue_priority = issue_comment[["priority"]][["name"]][[1]], + issue_affects_versions = stringi::stri_c(unlist(sapply(issue_comment[["versions"]],"[[","name")),collapse = ";"), + issue_fix_versions = stringi::stri_c(unlist(sapply(issue_comment[["fixVersions"]],"[[","name")),collapse = ";"), + issue_labels = stringi::stri_c(unlist(sapply(issue_comment[["labels"]],"[[",1)),collapse = ";"), + issue_votes = issue_comment[["votes"]][["votes"]][[1]], + issue_watchers = issue_comment[["watches"]][["watchCount"]][[1]], + + issue_created_datetimetz = issue_comment[["created"]][[1]], + issue_updated_datetimetz = issue_comment[["updated"]][[1]], + issue_resolution_datetimetz = issue_comment[["resolutiondate"]][[1]], + + issue_creator_id = issue_comment[["creator"]][["name"]][[1]], + issue_creator_name = issue_comment[["creator"]][["displayName"]][[1]], + issue_creator_timezone = issue_comment[["creator"]][["timeZone"]][[1]], + + issue_assignee_id = issue_comment[["assignee"]][["name"]][[1]], + issue_assignee_name = issue_comment[["assignee"]][["displayName"]][[1]], + issue_assignee_timezone = issue_comment[["assignee"]][["timeZone"]][[1]], + + issue_reporter_id = issue_comment[["reporter"]][["name"]][[1]], + issue_reporter_name = issue_comment[["reporter"]][["displayName"]][[1]], + issue_reporter_timezone = issue_comment[["reporter"]][["timeZone"]][[1]] + ) + # Comments + # For each issue, comment/comments contain 1 or more comments. Parse them + # in a separate table. + root_of_comments_list <- json_issue_comments[["issues"]][[i]][["fields"]][["comment"]] + # If root_of_comments_list does not exist, then this is an issue only json, skip parsing + if(length(root_of_comments_list) > 0){ + comments_list <- json_issue_comments[["issues"]][[i]][["fields"]][["comment"]][["comments"]] + # Even on a json with comments, some issues may not have comments, check if comments exist: + if(length(comments_list) > 0){ + # Parse all comments into issue_comments + issue_comments <- rbindlist(lapply(comments_list, + jira_parse_comment)) + # Add issue_key column to the start of the table + issue_comments <- cbind(data.table(issue_key=issue_key),issue_comments) + all_issues_comments[[i]] <- issue_comments + } + } + } - # Prepare two lists which will contain data.tables for all issues and all comments - # Both tables can share the issue_key, so they can be joined if desired. - all_issues <- list() - all_issues_comments <- list() - for(i in 1:n_issues){ + all_issues <- rbindlist(all_issues,fill=TRUE) + all_issues_comments <- rbindlist(all_issues_comments,fill=TRUE) - # The only use of "base_info" is to obtain the issue_key - issue_key <- json_issue_comments[["base_info"]][[i]][["key"]] + parsed_issues_comments <- list() + parsed_issues_comments[["issues"]] <- all_issues + parsed_issues_comments[["comments"]] <- all_issues_comments - # All other information is contained in "ext_info" - issue_comment <- json_issue_comments[["ext_info"]][[i]] + return(parsed_issues_comments) + } - # Parse all relevant *issue* fields - all_issues[[i]] <- data.table( - issue_key = issue_key, + issues_holder <- list() + comments_holder <- list() - issue_summary = issue_comment[["summary"]][[1]], - issue_type = issue_comment[["issuetype"]][["name"]][[1]], - issue_status = issue_comment[["status"]][["name"]][[1]], - issue_resolution = issue_comment[["resolution"]][["name"]][[1]], - issue_components = stringi::stri_c(unlist(sapply(issue_comment[["components"]],"[[","name")),collapse = ";"), - issue_description = issue_comment[["description"]], + for(filename in file_list){ + current_json <- paste0(json_folder_path, "/", filename) + parsed_data <- jira_parse_issues(current_json) + issues_holder <- append(issues_holder, list(parsed_data[["issues"]])) + comments_holder <- append(comments_holder, list(parsed_data[["comments"]])) + } - issue_created_datetimetz = issue_comment[["created"]][[1]], - issue_updated_datetimetz = issue_comment[["updated"]][[1]], - issue_resolution_datetimetz = issue_comment[["resolutiondate"]], + issues_holder <- rbindlist(issues_holder, fill=TRUE) + comments_holder <- rbindlist(comments_holder, fill=TRUE) - issue_creator_id = issue_comment[["creator"]][["name"]][[1]], - issue_creator_name = issue_comment[["creator"]][["displayName"]][[1]], - issue_creator_timezone = issue_comment[["creator"]][["timeZone"]][[1]], + return_info <- list() + return_info[["issues"]] <- issues_holder + return_info[["comments"]] <- comments_holder - issue_assignee_id = issue_comment[["assignee"]][["name"]][[1]], - issue_assignee_name = issue_comment[["assignee"]][["displayName"]][[1]], - issue_assignee_timezone = issue_comment[["assignee"]][["timeZone"]][[1]], + return(return_info) +} +#' Parse JIRA current issue +#' +#' Returns the file containing the most current issue in the specified folder. +#' +#' The folder assumes the following convention: "(PROJECTKEY)_(uniextimestamp_lowerbound)_(unixtimestamp_upperbound).json" +#' For example: "SAILUH_1231234_2312413.json". This nomenclature is defined by \code{\link{download_jira_issues}}. +#' +#' @param json_folder_path path to save folder containing JIRA issue and/or comments json files. +#' @return The name of the jira issue file with the latest created date that was created/downloaded for +#' use by the Jira downloader refresher +#' @export +#' @family parsers +parse_jira_latest_date <- function(json_folder_path){ + file_list <- list.files(json_folder_path) + time_list <- list() - issue_reporter_id = issue_comment[["reporter"]][["name"]][[1]], - issue_reporter_name = issue_comment[["reporter"]][["displayName"]][[1]], - issue_reporter_timezone = issue_comment[["reporter"]][["timeZone"]][[1]] - ) + # Checking if the save folder is empty + if (identical(file_list, character(0))){ + stop(stringi::stri_c("cannot open the connection")) + } - # Comments - # For each issue, comment/comments contain 1 or more comments. Parse them - # in a separate table. - root_of_comments_list <- json_issue_comments[["ext_info"]][[i]][["comment"]] - # If root_of_comments_list does not exist, then this is an issue only json, skip parsing - if(length(root_of_comments_list) > 0){ - comments_list <- json_issue_comments[["ext_info"]][[i]][["comment"]][["comments"]] - # Even on a json with comments, some issues may not have comments, check if comments exist: - if(length(comments_list) > 0){ - # Parse all comments into issue_comments - issue_comments <- rbindlist(lapply(comments_list, - jira_parse_comment)) - # Add issue_key column to the start of the table - issue_comments <- cbind(data.table(issue_key=issue_key),issue_comments) - all_issues_comments[[i]] <- issue_comments - } - } + for (j in file_list){ + j <- sub(".*_(\\w+)\\.[^.]+$", "\\1", j) + j <- as.numeric(j) + time_list <- append(time_list, j) } - all_issues <- rbindlist(all_issues,fill=TRUE) - all_issues_comments <- rbindlist(all_issues_comments,fill=TRUE) - parsed_issues_comments <- list() - parsed_issues_comments[["issues"]] <- all_issues - parsed_issues_comments[["comments"]] <- all_issues_comments + overall_latest_date <- as.character(max(unlist(time_list))) + + latest_issue_file <- grep(overall_latest_date, file_list, value = TRUE) - return(parsed_issues_comments) + return(latest_issue_file) } #' Format Parsed Jira to Replies #' @@ -196,57 +774,82 @@ parse_jira_rss_xml <- function(jira_issues_folderpath){ ############## Fake Generator ############## -#' Create JirAgileR Issue +#' Create JIRA Issue #' #' Creates a single JIRA Issue as a list, which can be saved as a JSON with or without comments. -#' Note the JSON follows the format used by JirAgileR package when downloading -#' JSONs, instead of the format specified by JIRA. #' #' @param jira_domain_url URL of JIRA domain (e.g. "https://project.org/jira") -#' @param issue_key issue key of JIRA issue (e.g. "PROJECT-68" or "GERONIMO-6723) +#' @param issue_key issue key of JIRA issue (e.g. "PROJECT-68" or "GERONIMO-6723") +#' @param project_key key of the project that contains the JIRA issue (e.g. "SPARK" or "GERONIMO") +#' @param summary summary of the issue (e.g. "Site Keeps Crashing") +#' @param description more detailed description of issue (e.g. "The program keeps crashing because this reason") #' @param issue_type type of JIRA issue (e.g. "New Feature", "Task", "Bug") -#' @param status status of issue for development (e.g. "In Progress") #' @param resolution name of resolution for issue (e.g. "Fixed") -#' @param title summary of the issue (e.g. "Site Keeps Crashing") -#' @param description more detailed description of issue (e.g. "The program keeps crashing because this reason") -#' @param components components of issue separate by ; (e.g. "x-core;x-spring") +#' @param priority the name of the priority of the issue (e.g. "Major", "Minor", "Trivial") +#' @param status status of issue for development (e.g. "In Progress") +#' @param labels the labels of the project (e.g. "message", "mail", "jira") +#' @param components list of components of issue (e.g. c("PS", "Tests")) +#' @param affects_versions list of affected versions (e.g. c("3.1.6", "4.1.0")) +#' @param fix_versions list of fixed versions (e.g. c("3.1.5", "4.0.0")) +#' @param assignee_name name of person the issue is being assigned to (e.g. "Joe Schmo") #' @param creator_name name of creator of issue (e.g. "John Doe") #' @param reporter_name name of reporter of issue (e.g. "Jane Doe") -#' @param assignee_name name of person the issue is being assigned to (e.g. "Joe Schmo") -#' @param comments character vector where each element is a comment string (e.g. c("This is first comment", "This is second comment")) -#' @return A list which represents the JirAgileR JSON in memory +#' @param comments character list where each element is a comment string (e.g. c("This is first comment", "This is second comment")) +#' @return A list which represents the JIRA JSON in memory #' @export #' @family {unittest} -make_jira_issue <- function(jira_domain_url, issue_key, issue_type, status, resolution, title, description, components, creator_name, reporter_name, assignee_name, comments = NULL) { - - # An issue in the JirAgileR format contains an `base_info_cell` - # and an `ext_info_cell`. This function calls the appropriate - # internal functions to define both blocks, and add - issues <- list() - - # Create `base_info_cell` - base_info_cell <- create_base_info(jira_domain_url, issue_key) - issues[["base_info"]][[1]] <- base_info_cell - - # Create `ext_info_cell` - ext_info_cell <- create_ext_info(jira_domain_url, issue_type, status, resolution, title, description, components, creator_name, reporter_name, assignee_name) +make_jira_issue <- function(jira_domain_url, issue_key, project_key, summary, description, issue_type, + resolution, priority, status, labels, components, affects_versions, fix_versions, + assignee_name, creator_name, reporter_name, comments = NULL) { + + # Create an issue with the given parameters as a list. If comments are specified, then add comments to the list + fields <- list( + parent = create_parent(jira_domain_url, issue_key, status, priority, issue_type), + fixVersions = create_fix_versions(jira_domain_url, fix_versions), + resolution = create_resolution(name = resolution), + priority = create_priority(jira_domain_url, priority), + labels = labels, + versions = create_versions(jira_domain_url, affects_versions), + assignee = create_assignee(jira_domain_url, assignee_name), + status = create_status(jira_domain_url, status), + components = create_components(jira_domain_url, components), + creator = create_creator(jira_domain_url, creator_name), + reporter = create_reporter(jira_domain_url, reporter_name), + votes = create_votes(jira_domain_url, issue_key), + issuetype = create_issue_type(jira_domain_url, issue_type), + project = create_project(jira_domain_url, project_key), + resolutiondate = "2007-08-13T19:12:33.000+0000", + watches = create_watches(jira_domain_url, issue_key), + created = "2007-07-08T06:07:06.000+0000", + updated = "2008-05-12T08:01:39.000+0000", + description = description, + summary = summary + ) - # Create `comment` if specified if (!is.null(comments) && length(comments) > 0) { - ext_info_cell[["comment"]][["comments"]] <- create_issue_comments(comments) - ext_info_cell[["comment"]][["maxResults"]] <- length(ext_info_cell[["comment"]][[1]]) - ext_info_cell[["comment"]][["total"]] <- length(ext_info_cell[["comment"]][[1]]) - ext_info_cell[["comment"]][["startAt"]] <- 0 + fields[["comment"]][["comments"]] <- create_issue_comments(comments) + fields[["comment"]][["maxResults"]] <- length(fields[["comment"]][[1]]) + fields[["comment"]][["total"]] <- length(fields[["comment"]][[1]]) + fields[["comment"]][["startAt"]] <- 0 } - issues[["ext_info"]][[1]] <- ext_info_cell + # generate a random id number + id <- sample(10000000: 99999999, 1) - #folder_path <- "/tmp" - #jira_json_path <- file.path(folder_path,"fake_issues.json") - #jsonlite::write_json(issues,file.path(folder_path,"fake_issues.json")) + # append the id to the Jira doman URL + self_url <- paste0(jira_domain_url, "/rest/api/2/issue", id) - return(issues) + # fill in the keys for the issue and append the 'fields' list + issue <- list( + expand = "schema, names", + id = as.character(id), + self = self_url, + key = issue_key, + fields = fields + ) + + return(issue) } @@ -260,244 +863,23 @@ make_jira_issue <- function(jira_domain_url, issue_key, issue_type, status, reso #' file name and extension. #' @return The `save_filepath` specified. #' @export -make_jira_issue_tracker <- function(issues,save_filepath) { +make_jira_issue_tracker <- function(issues, save_filepath) { # validate input if (!is.list(issues)) { stop("The issues parameter should be a list of issues.") } - issue_tracker <- list() - issue_tracker_base_list <- list() - issue_tracker_ext_list <- list() - - # Flatten function to format base_info properly for tracker format - flatten_base_info <- function(base_info) { - flattened_base_info <- list( - id = as.character(base_info[[1]][[1]]), - self = base_info[[1]][[2]], - key = base_info[[1]][[3]], - JirAgileR_id = as.numeric(base_info[[1]][[4]]) - ) - - # return final flattened base_info list - return(flattened_base_info) - } - - # Flatten functions to format ext_info properly for tracker format - flatten_ext_info <- function(ext_info) { - flattened_ext_info <- list() - - # title - flattened_ext_info$title <- ext_info[[1]][[1]][[1]] - - # issue_type - flatten_issuetype <- function(issuetype) { - flattened_issuetype <- list() - - flattened_issuetype$self <- issuetype[[1]][[1]][[1]] - flattened_issuetype$id <- as.character(issuetype[[2]][[1]][[1]]) - flattened_issuetype$description <- issuetype[[3]][[1]][[1]] - flattened_issuetype$iconUrl <- issuetype[[4]][[1]] - flattened_issuetype$name <- issuetype[[5]][[1]] - flattened_issuetype$subtask <- issuetype[[6]][[1]] - flattened_issuetype$avatarId <- issuetype[[7]][[1]] - - return(flattened_issuetype) - } - - flattened_ext_info$issuetype <- flatten_issuetype(ext_info[[1]][[2]]) - - # components - flatten_components <- function(components) { - flattened_components <- list() - - for(i in seq_along(components)) { - component <- components[[i]] - flattened_component <- list() - flattened_component$self <- component[[1]][[1]] - flattened_component$id <- component[[2]][[1]] - flattened_component$name <- component[[3]][[1]] - flattened_components[[i]] <- flattened_component - } - - return(flattened_components) - } - - flattened_ext_info$components <- flatten_components(ext_info[[1]][[3]]) - - # creator - flattened_ext_info$creator <- ext_info[[1]][[4]] - - # created - flattened_ext_info$created <- ext_info[[1]][[5]][[1]] - - # description - flattened_ext_info$description <- ext_info[[1]][[6]] - - # reporter - flatten_reporter <- function(reporter) { - flattened_reporter <- list() - flattened_reporter$self <- reporter[[1]][[1]] - flattened_reporter$name <- reporter[[2]][[1]] - flattened_reporter$key <- reporter[[3]][[1]] - flattened_reporter$avatarUrls <- reporter[[4]] - flattened_reporter$displayName <- reporter[[5]][[1]] - flattened_reporter$active <- reporter[[6]][[1]] - flattened_reporter$timeZone <- reporter[[7]][[1]] - - return(flattened_reporter) - } - - flattened_ext_info$reporter <- flatten_reporter(ext_info[[1]][[7]]) - - # resolution - flatten_resolution <- function(resolution) { - flattened_resolution <- list() - flattened_resolution$self <- resolution[[1]][[1]] - flattened_resolution$id <- resolution[[2]][[1]] - flattened_resolution$description <- resolution[[3]][[1]] - flattened_resolution$name <- resolution[[4]][[1]] - - return(flattened_resolution) - } - - flattened_ext_info$resolution <- flatten_resolution(ext_info[[1]][[8]]) - - #resolutiondate - flattened_ext_info$resolutiondate <- ext_info[[1]][[9]] - - # comments - if (length(ext_info[[1]]) >= 13) { - flattened_ext_info$comment <- ext_info[[1]][[13]] - } - - # assignee - flatten_assignee <- function(assignee) { - flattened_assignee <- list() - flattened_assignee$self <- assignee[[1]][[1]] - flattened_assignee$name <- assignee[[2]][[1]] - flattened_assignee$key <- assignee[[3]][[1]] - flattened_assignee$avatarUrls <- assignee[[4]] - flattened_assignee$displayName <- assignee[[5]][[1]] - flattened_assignee$active <- assignee[[6]][[1]] - flattened_assignee$timeZone <- assignee[[7]][[1]] - - return(flattened_assignee) - } - - flattened_ext_info$assignee <- flatten_reporter(ext_info[[1]][[10]]) - - # updated - flattened_ext_info$updated <- ext_info[[1]][[11]][[1]] - - # status - flatten_status <- function(status) { - flattened_status <- list() - flattened_status$self <-status[[1]][[1]] - flattened_status$description <- status[[2]][[1]] - flattened_status$iconUrl <- status[[3]][[1]] - flattened_status$name <- status[[4]] - flattened_status$id <- status[[5]][[1]] - - statusCategory <- list() - statusCategory$self <- status[[6]][[1]][[1]] - statusCategory$id <- status[[6]][[2]][[1]] - statusCategory$key <- status[[6]][[3]][[1]] - statusCategory$colorName <- status[[6]][[4]][[1]] - statusCategory$name <- status[[6]][[5]][[1]] - - flattened_status$statusCategory <- statusCategory - - return(flattened_status) - } - - flattened_ext_info$status <- flatten_status(ext_info[[1]][[12]]) - - # return final flattened ext_info list - return(flattened_ext_info) - } - - # loop through each issue - for(issue in issues) { - # get base & ext info for each issue - base_info <- flatten_base_info(issue[["base_info"]]) - ext_info <- flatten_ext_info(issue[["ext_info"]]) - - # combine each new base & ext list to respective list - issue_tracker_base_list <- c(issue_tracker_base_list, list(base_info)) - issue_tracker_ext_list <- c(issue_tracker_ext_list, list(ext_info)) - } - - # combine both lists to create final tracker - issue_tracker[["base_info"]] <- issue_tracker_base_list - issue_tracker[["ext_info"]] <- issue_tracker_ext_list - - jsonlite::write_json(issue_tracker,save_filepath) - - return(save_filepath) -} - - -#' Create base_info_cell -#' -#' Creates and formats base_info_cell list for \code{\link{make_jira_issue}}. -#' -#' @param jira_domain_url URL of JIRA domain -#' @param issue_key key for JIRA issue -#' @return A list named 'base_info_cell' containing all information for base cell in json file -#' @keywords internal -create_base_info <- function(jira_domain_url, issue_key) { - id <- sample(1:10, 1) - JirAgileR_id <- sample(1:10, 1) - - # Construct the issue API URL from the domain URL and issue key - issue_api_url <- paste0(jira_domain_url, "/rest/api/latest/issue/", id) - - base_info_cell <- list( - id = id, - self = issue_api_url, - key = issue_key, - JirAgileR_id = JirAgileR_id - ) - - return(base_info_cell) -} - -#' Create ext_info_cel -#' -#' Creates and formats ext_info_cell list \code{\link{make_jira_issue}}. -#' -#' @param jira_domain_url URL of JIRA domain -#' @param issue_type description of issue_type -#' @param status status of issue for development -#' @param resolution name of resolution for issue -#' @param title summary of the issue -#' @param description description of issue -#' @param components components of issue, a list with component names separated by ; (ex. "x-core;x-spring" is two components) -#' @param creator_name name of creator of issue -#' @param reporter_name name of reporter reporting the issue -#' @param assignee_name name of person the issue is being assigned to -#' @return A list named 'ext_info_cell' which contains all the parameters and its generated fake data formats -#' @keywords internal -create_ext_info <- function(jira_domain_url, issue_type, status, resolution, title, description, components, creator_name, reporter_name, assignee_name) { - - ext_info_cell <- list( - title = list(title), - issuetype = create_issue_type(jira_domain_url, issue_type), - components = create_components(jira_domain_url, components), - creator = create_creator(jira_domain_url, creator_name), - created = list("2007-07-08T06:07:06.000+0000"), - description = description, - reporter = create_reporter(jira_domain_url, reporter_name), - resolution = create_resolution(name = resolution), - resolutiondate = "2007-08-13T19:12:33.000+0000", - assignee = create_assignee(jira_domain_url, assignee_name), - updated = list("2008-05-12T08:01:39.000+0000"), - status = create_status(jira_domain_url, status) + export_issues <- list( + expand = "schema,names", + startAt = 0, + maxResults = 50, + total = length(issues), + issues = issues ) - return(ext_info_cell) + jsonlite::write_json(export_issues, save_filepath, auto_unbox=TRUE) + return(save_filepath) } #' Create Issue Comments @@ -506,10 +888,11 @@ create_ext_info <- function(jira_domain_url, issue_type, status, resolution, tit #' Other parameters associated to the comments, such as the author #' and update author are currently hardcoded. #' -#' @param comments A character vector containing the comment body. +#' @param comments A character list containing the comment body. +#' @return A list named 'comments_list' that has a list of comments #' @keywords internal create_issue_comments <- function(comments) { - comments_vector <- list() + comments_list <- list() # go through and make comment for each body in comment_bodies # only comment bodies changes for comments, the rest of comments information is hard coded below @@ -549,15 +932,16 @@ create_issue_comments <- function(comments) { created = "2021-01-01T10:00:00.000+0000", updated = "2021-01-01T12:00:00.000+0000" ) - comments_vector[[length(comments_vector) + 1]] <- comment + comments_list[[length(comments_list) + 1]] <- comment } - return(comments_vector) + return(comments_list) } #' Create Issue Type #' -#' Create issue type cell for \code{\link{make_jira_issue}}. +#' Create issue type cell for \code{\link{make_jira_issue}}. This represents the 'Type' +#' label in JIRA #' #' @param jira_domain_url URL of JIRA domain #' @param issue_type name of the issue type (e.g. New Feature) @@ -569,13 +953,13 @@ create_issue_type <- function(jira_domain_url, issue_type) { self_url <- paste0(jira_domain_url, "/rest/api/", issue_id, "/issuetype/", issue_id) issue_type <- list( - self = list(list(self_url)), - id = list(list(issue_id)), - description = list(list("A new feature of the product, which has yet to be developed.")), - iconUrl = list("https://domain.org/jira/secure/viewavatar?size=xsmall&avatarId=21141&avatarType=issuetype"), - name = list(issue_type), - subtask = list(FALSE), - avatarId = list(21141) + self = self_url, + id = issue_id, + description = "A new feature of the product, which has yet to be developed.", + iconUrl = "https://domain.org/jira/secure/viewavatar?size=xsmall&avatarId=21141&avatarType=issuetype", + name = issue_type, + subtask = FALSE, + avatarId = 21141 ) return(issue_type) @@ -586,24 +970,24 @@ create_issue_type <- function(jira_domain_url, issue_type) { #' Creates the component cells for \code{\link{make_jira_issue}}. #' #' @param jira_domain_url URL of JIRA domain -#' @param components string of names of components (ex. "x-core;x-spring" is two components) +#' @param components list of names of components #' @return A list named 'components' which contains each component and its details #' @keywords internal create_components <- function(jira_domain_url, components) { - - # separate components names with ; (ex. "x-core;x-spring" is two components) - components_names <- unlist(stringi::stri_split_regex(components, pattern = ";")) components_list <- list() # for loop to create a component for each component name - for (name in components_names) { - id <- sample(10000000: 99999999, 1) + for (name in components) { + + id <- sample(10000: 99999999, 1) + self_url <- paste0(jira_domain_url, "/rest/api/2/component/", id) component <- list( - self = list(self_url), - id = list(as.character(id)), - name = list(name) + self = self_url, + id = as.character(id), + name = name, + description = "This is the description for the component" ) # add component to list which will be returned at the end @@ -664,13 +1048,13 @@ create_reporter <- function(jira_domain_url, reporter_name) { ) reporter <- list( - self = list(self_url), + self = self_url, name = "user_id", key = "user_id", avatarUrls = avatarUrls, - displayName = list(reporter_name), - active = list(TRUE), - timeZone = list("Etc/UTC") + displayName = reporter_name, + active = TRUE, + timeZone = "Etc/UTC" ) return(reporter) @@ -691,10 +1075,10 @@ create_resolution <- function(self_url = "https://domain.org/jira/rest/api/2/res description = "A fix for this issue is checked into the tree and tested.", name = "Fixed") { resolution <- list( - self = list(self_url), - id = list(id), - description = list(description), - name = list(name) + self = self_url, + id = id, + description = description, + name = name ) return(resolution) @@ -702,7 +1086,7 @@ create_resolution <- function(self_url = "https://domain.org/jira/rest/api/2/res #' Create Assignee #' -#' Creates a assignee cell for \code{\link{make_jira_issue}}. +#' Creates an assignee cell for \code{\link{make_jira_issue}}. #' #' @param jira_domain_url URL of JIRA domain #' @param assignee_name name of assignee @@ -720,13 +1104,13 @@ create_assignee <- function(jira_domain_url, assignee_name) { ) assignee <- list( - self = list(self_url), + self = self_url, name = "user_id", key = "user_id", avatarUrls = avatarUrls, - displayName = list(assignee_name), - active = list(TRUE), - timeZone = list("Etc/UTC") + displayName = assignee_name, + active = TRUE, + timeZone = "Etc/UTC" ) return(assignee) @@ -738,7 +1122,7 @@ create_assignee <- function(jira_domain_url, assignee_name) { #' #' @param jira_domain_url URL of JIRA domain #' @param status description of status -#' @return A list named 'status' containing status's information +#' @return A list named 'status' containing the status of the issue #' @keywords internal create_status <- function(jira_domain_url, status) { @@ -749,19 +1133,217 @@ create_status <- function(jira_domain_url, status) { statusCategory_self_url <- paste0(jira_domain_url, "/rest/api/2/statuscategory/", status_category_id) status <- list( - self = list(self_url), - description = list("The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened."), - iconUrl = list("https://domain.org/jira/images/icons/statuses/closed.png"), + self = self_url, + description = "The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.", + iconUrl = "https://domain.org/jira/images/icons/statuses/closed.png", name = status, id = as.character(status_id), statusCategory = list( - self = list(statusCategory_self_url), - id = list(status_category_id), - key = list("done"), - colorName = list("green"), - name = list("Done") + self = statusCategory_self_url, + id = status_category_id, + key = "done", + colorName = "green", + name = "Done" ) ) return(status) } + +#' Create Fix Version +#' +#' Create a fixVersions cell for \code{\link{make_jira_issue}}. This represents the +#' 'Fixed Version/s' label in JIRA +#' +#' @param jira_domain_url URL of JIRA domain +#' @param fix_versions list of fixed versions for the issue +#' @return A list named 'fixVersions' with a list of fixed versions and version information +#' @keywords internal +create_fix_versions <- function(jira_domain_url, fix_versions) { + + fixVersions_list <- list() + + for(fix_version in fix_versions){ + id <- sample(10000000: 99999999, 1) + self_url <- paste0(jira_domain_url, "/rest/api/2/version/", id) + + version <- list( + self = self_url, + id = as.character(id), + description = "This is a description of the fixVersion", + name = fix_version, + archived = FALSE, + released = TRUE, + releaseDate = "2021-01-01T10:00:00.000+0000" + ) + + fixVersions_list[[length(fixVersions_list) + 1]] <- version + } + + return(fixVersions_list) +} + +#' Create Priority +#' +#' Create a priority cell for \code{\link{make_jira_issue}}. +#' +#' @param jira_domain_url URL of JIRA domain +#' @param priority the name of the priority of the issue (Major, Minor, Trivial) +#' @return A list named 'priority' containing the priority of the issue +#' @keywords internal +create_priority <- function(jira_domain_url, priority) { + + id <- sample(1:10, 1) + + self_url <- paste0(jira_domain_url, "/rest/api/2/priority/", id) + + priority <- list( + self = self_url, + iconUrl = "https://issues.apache.org/jira/images/icons/priorities/major.svg", + name = priority, + id = as.character(id) + ) + + return(priority) +} + +#' Create Parent +#' +#' Create a parent cell for \code{\link{make_jira_issue}}. Currently, the parent has the same +#' issue_key, status, priority, and issue_type as the base issue +#' +#' @param jira_domain_url URL of JIRA domain +#' @param parent_issue_key issue key of the parent issue of the current JIRA issue +#' @param status status of issue for development +#' @param priority the name of the priority of the issue +#' @param issue_type type of JIRA issue +#' @return A list named 'parent' that contains a parent issue and it's fields +#' @keywords internal +create_parent <- function(jira_domain_url, issue_key, status, priority, issue_type) { + + id <- sample(10000000: 99999999, 1) + + self_url <- paste0(jira_domain_url, "/rest/api/2/issue/", id) + + fields <- list( + summary = "This is a summary", + status = create_status(jira_domain_url, status), + priority = create_priority(jira_domain_url, priority), + issuetype = create_issue_type(jira_domain_url, issue_type) + ) + + parent <- list( + id = as.character(id), + key = issue_key, + self = self_url, + fields = fields + ) + + return(parent) +} + +#' Create Project +#' +#' Create a project cell for \code{\link{make_jira_issue}}. +#' +#' @param jira_domain_url URL of JIRA domain +#' @param project_key key of the project that contains the JIRA issue (e.g. "SPARK" or "GERONIMO") +#' @return A list named 'project' that contains the project's information +#' @keywords internal +create_project <- function(jira_domain_url, project_key) { + + id <- sample(10000000: 99999999, 1) + + self_url <- paste0(jira_domain_url, "/rest/api/2/project/", id) + + avatarUrls = list( + "48x48" = "https://example.com/jira/secure/useravatar?size=large&ownerId=user1", + "24x24" = "https://example.com/jira/secure/useravatar?size=small&ownerId=user1", + "16x16" = "https://example.com/jira/secure/useravatar?size=xsmall&ownerId=user1", + "32x32" = "https://example.com/jira/secure/useravatar?size=medium&ownerId=user1" + ) + + project <- list( + self = self_url, + id = as.character(id), + key = project_key, + name = project_key, + projectTypeKey = "software", + avartarUrls = avatarUrls + ) +} + +#' Create Versions +#' +#' Create a versions cell for \code{\link{make_jira_issue}}. This cell represents +#' the 'Affects Version/s' label in JIRA +#' +#' @param jira_domain_url URL of JIRA domain +#' @param affects_versions list of version names for the issue +#' @return A list named 'versions' with a list of versions +#' @keywords internal +create_versions <- function(jira_domain_url, affects_versions) { + + versions_list <- list() + + for(affects_version in affects_versions){ + id <- sample(10000000: 99999999, 1) + self_url <- paste0(jira_domain_url, "/rest/api/2/version/", id) + + version <- list( + self = self_url, + id = as.character(id), + description = "This is a description of the version", + name = affects_version, + archived = FALSE, + released = TRUE, + releaseDate = "2024-01-01T10:00:00.000+0000" + ) + + versions_list[[length(versions_list) + 1]] <- version + } + + return(versions_list) +} + +#' Create Votes +#' +#' Create a votes cell for \code{\link{make_jira_issue}}. +#' +#' @param jira_domain_url URL of JIRA domain +#' @param issue_key issue key of JIRA issue +#' @return A list named 'votes' that has the number of votes for the issue +#' @keywords internal +create_votes <- function(jira_domain_url, issue_key) { + + self_url <- paste0(jira_domain_url, "/rest/api/2/issue/", issue_key, "votes") + + votes <- list( + self = self_url, + votes = 10, + hasVoted = FALSE + ) + + return(votes) +} + +#' Create Watches +#' +#' Create a watches cell for \code{\link{make_jira_issue}}. +#' +#' @param jira_domain_url URL of JIRA domain +#' @param issue_key issue key of JIRA issue +#' @return A list named 'watches' that has the number of watchers for the issue +#' @keywords internal +create_watches <- function(jira_domain_url, issue_key) { + + self_url <- paste0(jira_domain_url, "/rest/api/2/issue/", issue_key, "watchers") + + watches <- list( + self = self_url, + watchCount = 15, + isWatching = FALSE + ) + + return(watches) +} diff --git a/_pkgdown.yml b/_pkgdown.yml index 1e261995..98b851a8 100644 --- a/_pkgdown.yml +++ b/_pkgdown.yml @@ -110,10 +110,15 @@ reference: Notebook for details. - contents: - parse_jira + - parse_jira_latest_date - parse_jira_replies - parse_jira_rss_xml - make_jira_issue - make_jira_issue_tracker + - download_jira_issues_comments + - download_jira_issues_comments_by_date + - download_jira_issues_comments_by_issuekey + - refresh_jira_issues_comments_by_issuekey - title: __GitHub__ desc: > Functions to interact and download data from GitHub API. diff --git a/conf/geronimo.yml b/conf/geronimo.yml index 99ee59a9..56841607 100644 --- a/conf/geronimo.yml +++ b/conf/geronimo.yml @@ -58,8 +58,8 @@ issue_tracker: domain: https://issues.apache.org/jira project_key: GERONIMO # Download using `download_jira_data.Rmd` - issues: ../../rawdata/issue_tracker/geronimo_issues.json - issue_comments: ../../rawdata/issue_tracker/geronimo_issue_comments.json + issues: ../../rawdata/issue_tracker/geronimo/issues/ + issue_comments: ../../rawdata/issue_tracker/geronimo/issue_comments/ github: # Obtained from the project's GitHub URL owner: apache diff --git a/conf/kaiaulu.yml b/conf/kaiaulu.yml index a5f007cc..3d0e8e9b 100644 --- a/conf/kaiaulu.yml +++ b/conf/kaiaulu.yml @@ -55,11 +55,11 @@ mailing_list: issue_tracker: jira: # Obtained from the project's JIRA URL - #domain: https://issues.apache.org/jira - #project_key: GERONIMO + domain: https://sailuh.atlassian.net + project_key: SAILUH # Download using `download_jira_data.Rmd` - #issues: ../../rawdata/issue_tracker/geronimo_issues.json - #issue_comments: ../../rawdata/issue_tracker/geronimo_issue_comments.json + issues: ../../rawdata/issue_tracker/kaiaulu/issues/ + issue_comments: ../../rawdata/issue_tracker/kaiaulu/issue_comments/ github: # Obtained from the project's GitHub URL owner: sailuh diff --git a/man/create_assignee.Rd b/man/create_assignee.Rd index 0e6bce1a..1b24ffd2 100644 --- a/man/create_assignee.Rd +++ b/man/create_assignee.Rd @@ -15,6 +15,6 @@ create_assignee(jira_domain_url, assignee_name) A list named 'assignee' which contains the assignee's information } \description{ -Creates a assignee cell for \code{\link{make_jira_issue}}. +Creates an assignee cell for \code{\link{make_jira_issue}}. } \keyword{internal} diff --git a/man/create_base_info.Rd b/man/create_base_info.Rd deleted file mode 100644 index 59103c39..00000000 --- a/man/create_base_info.Rd +++ /dev/null @@ -1,20 +0,0 @@ -% Generated by roxygen2: do not edit by hand -% Please edit documentation in R/jira.R -\name{create_base_info} -\alias{create_base_info} -\title{Create base_info_cell} -\usage{ -create_base_info(jira_domain_url, issue_key) -} -\arguments{ -\item{jira_domain_url}{URL of JIRA domain} - -\item{issue_key}{key for JIRA issue} -} -\value{ -A list named 'base_info_cell' containing all information for base cell in json file -} -\description{ -Creates and formats base_info_cell list for \code{\link{make_jira_issue}}. -} -\keyword{internal} diff --git a/man/create_components.Rd b/man/create_components.Rd index 41ec53df..b1b43b8f 100644 --- a/man/create_components.Rd +++ b/man/create_components.Rd @@ -9,7 +9,7 @@ create_components(jira_domain_url, components) \arguments{ \item{jira_domain_url}{URL of JIRA domain} -\item{components}{string of names of components (ex. "x-core;x-spring" is two components)} +\item{components}{list of names of components} } \value{ A list named 'components' which contains each component and its details diff --git a/man/create_ext_info.Rd b/man/create_ext_info.Rd deleted file mode 100644 index 285deef9..00000000 --- a/man/create_ext_info.Rd +++ /dev/null @@ -1,47 +0,0 @@ -% Generated by roxygen2: do not edit by hand -% Please edit documentation in R/jira.R -\name{create_ext_info} -\alias{create_ext_info} -\title{Create ext_info_cel} -\usage{ -create_ext_info( - jira_domain_url, - issue_type, - status, - resolution, - title, - description, - components, - creator_name, - reporter_name, - assignee_name -) -} -\arguments{ -\item{jira_domain_url}{URL of JIRA domain} - -\item{issue_type}{description of issue_type} - -\item{status}{status of issue for development} - -\item{resolution}{name of resolution for issue} - -\item{title}{summary of the issue} - -\item{description}{description of issue} - -\item{components}{components of issue, a list with component names separated by ; (ex. "x-core;x-spring" is two components)} - -\item{creator_name}{name of creator of issue} - -\item{reporter_name}{name of reporter reporting the issue} - -\item{assignee_name}{name of person the issue is being assigned to} -} -\value{ -A list named 'ext_info_cell' which contains all the parameters and its generated fake data formats -} -\description{ -Creates and formats ext_info_cell list \code{\link{make_jira_issue}}. -} -\keyword{internal} diff --git a/man/create_fix_versions.Rd b/man/create_fix_versions.Rd new file mode 100644 index 00000000..35f8fe12 --- /dev/null +++ b/man/create_fix_versions.Rd @@ -0,0 +1,21 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{create_fix_versions} +\alias{create_fix_versions} +\title{Create Fix Version} +\usage{ +create_fix_versions(jira_domain_url, fix_versions) +} +\arguments{ +\item{jira_domain_url}{URL of JIRA domain} + +\item{fix_versions}{list of fixed versions for the issue} +} +\value{ +A list named 'fixVersions' with a list of fixed versions and version information +} +\description{ +Create a fixVersions cell for \code{\link{make_jira_issue}}. This represents the +'Fixed Version/s' label in JIRA +} +\keyword{internal} diff --git a/man/create_issue_comments.Rd b/man/create_issue_comments.Rd index 9c47c6d8..fd0633fa 100644 --- a/man/create_issue_comments.Rd +++ b/man/create_issue_comments.Rd @@ -7,7 +7,10 @@ create_issue_comments(comments) } \arguments{ -\item{comments}{A character vector containing the comment body.} +\item{comments}{A character list containing the comment body.} +} +\value{ +A list named 'comments_list' that has a list of comments } \description{ Create issue comments cell for \code{\link{make_jira_issue}}. diff --git a/man/create_issue_type.Rd b/man/create_issue_type.Rd index 23be7c09..29f563f2 100644 --- a/man/create_issue_type.Rd +++ b/man/create_issue_type.Rd @@ -15,6 +15,7 @@ create_issue_type(jira_domain_url, issue_type) A list named 'issue_type' that represents the issue type of the JIRA issue } \description{ -Create issue type cell for \code{\link{make_jira_issue}}. +Create issue type cell for \code{\link{make_jira_issue}}. This represents the 'Type' +label in JIRA } \keyword{internal} diff --git a/man/create_parent.Rd b/man/create_parent.Rd new file mode 100644 index 00000000..c4e0e926 --- /dev/null +++ b/man/create_parent.Rd @@ -0,0 +1,27 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{create_parent} +\alias{create_parent} +\title{Create Parent} +\usage{ +create_parent(jira_domain_url, issue_key, status, priority, issue_type) +} +\arguments{ +\item{jira_domain_url}{URL of JIRA domain} + +\item{status}{status of issue for development} + +\item{priority}{the name of the priority of the issue} + +\item{issue_type}{type of JIRA issue} + +\item{parent_issue_key}{issue key of the parent issue of the current JIRA issue} +} +\value{ +A list named 'parent' that contains a parent issue and it's fields +} +\description{ +Create a parent cell for \code{\link{make_jira_issue}}. Currently, the parent has the same +issue_key, status, priority, and issue_type as the base issue +} +\keyword{internal} diff --git a/man/create_priority.Rd b/man/create_priority.Rd new file mode 100644 index 00000000..e2ec7520 --- /dev/null +++ b/man/create_priority.Rd @@ -0,0 +1,20 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{create_priority} +\alias{create_priority} +\title{Create Priority} +\usage{ +create_priority(jira_domain_url, priority) +} +\arguments{ +\item{jira_domain_url}{URL of JIRA domain} + +\item{priority}{the name of the priority of the issue (Major, Minor, Trivial)} +} +\value{ +A list named 'priority' containing the priority of the issue +} +\description{ +Create a priority cell for \code{\link{make_jira_issue}}. +} +\keyword{internal} diff --git a/man/create_project.Rd b/man/create_project.Rd new file mode 100644 index 00000000..e6ba3e0a --- /dev/null +++ b/man/create_project.Rd @@ -0,0 +1,20 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{create_project} +\alias{create_project} +\title{Create Project} +\usage{ +create_project(jira_domain_url, project_key) +} +\arguments{ +\item{jira_domain_url}{URL of JIRA domain} + +\item{project_key}{key of the project that contains the JIRA issue (e.g. "SPARK" or "GERONIMO")} +} +\value{ +A list named 'project' that contains the project's information +} +\description{ +Create a project cell for \code{\link{make_jira_issue}}. +} +\keyword{internal} diff --git a/man/create_status.Rd b/man/create_status.Rd index 8989c069..a5c08bab 100644 --- a/man/create_status.Rd +++ b/man/create_status.Rd @@ -12,7 +12,7 @@ create_status(jira_domain_url, status) \item{status}{description of status} } \value{ -A list named 'status' containing status's information +A list named 'status' containing the status of the issue } \description{ Creates a status cell for \code{\link{make_jira_issue}}. diff --git a/man/create_versions.Rd b/man/create_versions.Rd new file mode 100644 index 00000000..a9bac2b9 --- /dev/null +++ b/man/create_versions.Rd @@ -0,0 +1,21 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{create_versions} +\alias{create_versions} +\title{Create Versions} +\usage{ +create_versions(jira_domain_url, affects_versions) +} +\arguments{ +\item{jira_domain_url}{URL of JIRA domain} + +\item{affects_versions}{list of version names for the issue} +} +\value{ +A list named 'versions' with a list of versions +} +\description{ +Create a versions cell for \code{\link{make_jira_issue}}. This cell represents +the 'Affects Version/s' label in JIRA +} +\keyword{internal} diff --git a/man/create_votes.Rd b/man/create_votes.Rd new file mode 100644 index 00000000..0c499785 --- /dev/null +++ b/man/create_votes.Rd @@ -0,0 +1,20 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{create_votes} +\alias{create_votes} +\title{Create Votes} +\usage{ +create_votes(jira_domain_url, issue_key) +} +\arguments{ +\item{jira_domain_url}{URL of JIRA domain} + +\item{issue_key}{issue key of JIRA issue} +} +\value{ +A list named 'votes' that has the number of votes for the issue +} +\description{ +Create a votes cell for \code{\link{make_jira_issue}}. +} +\keyword{internal} diff --git a/man/create_watches.Rd b/man/create_watches.Rd new file mode 100644 index 00000000..a671864a --- /dev/null +++ b/man/create_watches.Rd @@ -0,0 +1,20 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{create_watches} +\alias{create_watches} +\title{Create Watches} +\usage{ +create_watches(jira_domain_url, issue_key) +} +\arguments{ +\item{jira_domain_url}{URL of JIRA domain} + +\item{issue_key}{issue key of JIRA issue} +} +\value{ +A list named 'watches' that has the number of watchers for the issue +} +\description{ +Create a watches cell for \code{\link{make_jira_issue}}. +} +\keyword{internal} diff --git a/man/download_jira_issues.Rd b/man/download_jira_issues.Rd new file mode 100644 index 00000000..f2552b69 --- /dev/null +++ b/man/download_jira_issues.Rd @@ -0,0 +1,80 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{download_jira_issues} +\alias{download_jira_issues} +\title{Download JIRA Issues and/or Comments} +\usage{ +download_jira_issues( + domain, + jql_query, + fields, + username = NULL, + password = NULL, + save_folder_path, + max_results = 50, + max_total_downloads = 5000, + search_query = NULL, + verbose = FALSE +) +} +\arguments{ +\item{domain}{Custom JIRA domain URL} + +\item{jql_query}{Specific query string to specify criteria for fetching} + +\item{fields}{List of fields that are downloaded in each issue} + +\item{username}{the JIRA username} + +\item{password}{the JIRA password/API token} + +\item{save_folder_path}{Path that files will be saved in .json format} + +\item{max_results}{(optional) the [maximum number of results](https://confluence.atlassian.com/jirakb/how-to-use-the-maxresults-api-parameter-for-jira-issue-search-rest-api-1251999998.html) +to download per page. Default is 50.} + +\item{max_total_downloads}{Maximum downloads per function call. +This parameter specifies how many issues should be downloaded. Subsequent API calls will stop if they +reach or surpass this value.} + +\item{search_query}{an optional API parameter that alters the GET request. +See \code{\link{download_jira_issues_by_date}} and \code{\link{download_jira_issues_by_issue_key}} +source code for examples.} + +\item{verbose}{Set verbose=TRUE to print execution details.} +} +\description{ +Download JIRA issues and/or comments using [rest/api/2/search](https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issue-search/#api-rest-api-2-search-post) endpoint +to the specified `save_folder_path`. +} +\details{ +The folder assumes the following convention: "(PROJECTKEY)_(uniextimestamp_lowerbound)_(unixtimestamp_upperbound).json" +For example: "SAILUH_1231234_2312413.json". + +Comments data are added when the field `comment` is included. + +If a project requires authentication and authentication fails, the function will end without downloading any data. + +If the number of results per page returned is less than the number specified, the max_results value will adjust to that value. +} +\seealso{ +\code{\link{download_jira_issues_by_date}} to download JIRA issues and/or comments by date range + +\code{\link{download_jira_issues_by_issue_key}} to download JIRA issues and/or comments by issue key range + +\code{\link{parse_jira}} to parse jira data with or without comments + +\code{\link{refresh_jira_issues}} to obtain more recent data from any of the downloader functions + +Other jira: +\code{\link{download_jira_issues_by_date}()}, +\code{\link{download_jira_issues_by_issue_key}()}, +\code{\link{refresh_jira_issues}()} + +Other downloaders: +\code{\link{download_jira_issues_by_date}()}, +\code{\link{download_jira_issues_by_issue_key}()}, +\code{\link{refresh_jira_issues}()} +} +\concept{downloaders} +\concept{jira} diff --git a/man/download_jira_issues_by_date.Rd b/man/download_jira_issues_by_date.Rd new file mode 100644 index 00000000..697fdb48 --- /dev/null +++ b/man/download_jira_issues_by_date.Rd @@ -0,0 +1,85 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{download_jira_issues_by_date} +\alias{download_jira_issues_by_date} +\title{Download JIRA Issues and/or Comments by Date Range} +\usage{ +download_jira_issues_by_date( + domain, + jql_query, + fields, + username = NULL, + password = NULL, + save_folder_path, + max_results, + max_total_downloads, + date_lower_bound = NULL, + date_upper_bound = NULL, + verbose +) +} +\arguments{ +\item{domain}{Custom JIRA domain URL} + +\item{jql_query}{Specific query string to specify criteria for fetching} + +\item{fields}{List of fields that are downloaded in each issue} + +\item{username}{the JIRA username} + +\item{password}{the JIRA password/API token} + +\item{save_folder_path}{Path that files will be saved in .json format} + +\item{max_results}{(optional) the [maximum number of results](https://confluence.atlassian.com/jirakb/how-to-use-the-maxresults-api-parameter-for-jira-issue-search-rest-api-1251999998.html) +to download per page. Default is 50.} + +\item{max_total_downloads}{Maximum downloads per function call. +This parameter specifies how many issues should be downloaded. Subsequent API calls will stop if they +reach or surpass this value. +See \code{\link{download_jira_issues_by_date}} and \code{\link{download_jira_issues_by_issue_key}} +source code for examples.} + +\item{date_lower_bound}{Optional. Specify the lower bound date time (e.g. 2023/11/16 21:00)} + +\item{date_upper_bound}{Optional. Specify the upper bound date time (e.g. 2023/11/17 21:00)} + +\item{verbose}{Set verbose=TRUE to print execution details.} +} +\description{ +Wraps around \code{\link{download_jira_issues}} providing JQL query parameters for specifying date ranges. +Only issues created in the specified date range will be downloaded. +} +\details{ +Acceptable formats for `date_lower_bound` and `date_upper_bound` are: + +* "yyyy/MM/dd HH:mm" +* "yyyy-MM-dd HH:mm" +* "yyyy/MM/dd" +* "yyyy-MM-dd" + +For example: `date_lower_bound=2023/11/16 21:00` (an issue ocurring at the exact specified time will also be downloaded). + +For further details on the `created` JQL Query see [the associated JIRA API documentation](https://support.atlassian.com/jira-software-cloud/docs/jql-fields/#Created). +} +\seealso{ +\code{\link{download_jira_issues_by_issue_key}} to download JIRA issues and/or comments by issue key range + +\code{\link{download_jira_issues}} for more flexibility in specifying the JQL query + +\code{\link{parse_jira}} to parse jira data with or without comments + +\code{\link{refresh_jira_issues}} to obtain more recent data from any of the downloader functions + +Other jira: +\code{\link{download_jira_issues_by_issue_key}()}, +\code{\link{download_jira_issues}()}, +\code{\link{refresh_jira_issues}()} + +Other downloaders: +\code{\link{download_jira_issues_by_issue_key}()}, +\code{\link{download_jira_issues}()}, +\code{\link{refresh_jira_issues}()} +} +\concept{downloaders} +\concept{jira} diff --git a/man/download_jira_issues_by_issue_key.Rd b/man/download_jira_issues_by_issue_key.Rd new file mode 100644 index 00000000..b452878f --- /dev/null +++ b/man/download_jira_issues_by_issue_key.Rd @@ -0,0 +1,80 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{download_jira_issues_by_issue_key} +\alias{download_jira_issues_by_issue_key} +\title{Download JIRA Issues and/or Comments by Issue Key Range} +\usage{ +download_jira_issues_by_issue_key( + domain, + jql_query, + fields, + username = NULL, + password = NULL, + save_folder_path, + max_results, + max_total_downloads, + issue_key_lower_bound = NULL, + issue_key_upper_bound = NULL, + verbose +) +} +\arguments{ +\item{domain}{Custom JIRA domain URL} + +\item{jql_query}{Specific query string to specify criteria for fetching} + +\item{fields}{List of fields that are downloaded in each issue} + +\item{username}{the JIRA username} + +\item{password}{the JIRA password/API token} + +\item{save_folder_path}{Path that files will be saved in .json format} + +\item{max_results}{(optional) the [maximum number of results](https://confluence.atlassian.com/jirakb/how-to-use-the-maxresults-api-parameter-for-jira-issue-search-rest-api-1251999998.html) +to download per page. Default is 50.} + +\item{max_total_downloads}{Maximum downloads per function call. +This parameter specifies how many issues should be downloaded. Subsequent API calls will stop if they +reach or surpass this value. +See \code{\link{download_jira_issues_by_date}} and \code{\link{download_jira_issues_by_issue_key}} +source code for examples.} + +\item{issue_key_lower_bound}{Optional. Specify the lower bound issue key (e.g. SAILUH-1)} + +\item{issue_key_upper_bound}{Optional. Specify the upper bound issue key (e.g. SAILUH-3)} + +\item{verbose}{Set verbose=TRUE to print execution details.} +} +\description{ +#' Wraps around \code{\link{download_jira_issues}} providing jql query parameters for specifying issue key ranges. +Only issues created in the specified date range inclusive will be downloaded. +} +\details{ +The acceptable format for `issue_key_lower_bound` and `issue_key_upper_bound` is: - + +For example: `issue_key_lower_bound=SAILUH-1` (SAILUH-1 will also be downloaded). + +For further details on the `issueKey` JQL Query see [the associated JIRA API documentation](https://support.atlassian.com/jira-software-cloud/docs/jql-fields/#Issue-key) +} +\seealso{ +\code{\link{download_jira_issues_by_date}} to download JIRA issues and/or comments by date range + +\code{\link{download_jira_issues}} for more flexibility in specifying the JQL query + +\code{\link{parse_jira}} to parse jira data with or without comments + +\code{\link{refresh_jira_issues}} to obtain more recent data from any of the downloader functions + +Other jira: +\code{\link{download_jira_issues_by_date}()}, +\code{\link{download_jira_issues}()}, +\code{\link{refresh_jira_issues}()} + +Other downloaders: +\code{\link{download_jira_issues_by_date}()}, +\code{\link{download_jira_issues}()}, +\code{\link{refresh_jira_issues}()} +} +\concept{downloaders} +\concept{jira} diff --git a/man/example_jira_issue_comments.Rd b/man/example_jira_issue_comments.Rd index b5556af1..33df571d 100644 --- a/man/example_jira_issue_comments.Rd +++ b/man/example_jira_issue_comments.Rd @@ -12,7 +12,7 @@ example_jira_issue_comments(folder_path = "/tmp", folder_name) \item{folder_name}{The name of the folder} } \value{ -the JSON path of the newly created issue issue tracker +the JSON folder path of the newly created issue issue tracker } \description{ Create fake jira issue tracker with one issue with 2 comments diff --git a/man/example_jira_issue_components.Rd b/man/example_jira_issue_components.Rd index fd2da078..f861ac68 100644 --- a/man/example_jira_issue_components.Rd +++ b/man/example_jira_issue_components.Rd @@ -12,7 +12,7 @@ example_jira_issue_components(folder_path = "/tmp", folder_name) \item{folder_name}{The name of the folder} } \value{ -the JSON path of the newly created issue issue tracker +the JSON folder path of the newly created issue issue tracker } \description{ This example can be used to evaluate the parser does not replicate diff --git a/man/example_jira_two_issues.Rd b/man/example_jira_two_issues.Rd index cef8ff17..d6021be0 100644 --- a/man/example_jira_two_issues.Rd +++ b/man/example_jira_two_issues.Rd @@ -12,7 +12,7 @@ example_jira_two_issues(folder_path = "/tmp", folder_name) \item{folder_name}{The name of the folder} } \value{ -the JSON path of the newly created issue issue tracker +the JSON folder path of the newly created issue issue tracker } \description{ Create fake JIRA issue tracker with 2 issues, no comments diff --git a/man/make_jira_issue.Rd b/man/make_jira_issue.Rd index e3fc3d7d..6748fe99 100644 --- a/man/make_jira_issue.Rd +++ b/man/make_jira_issue.Rd @@ -2,55 +2,68 @@ % Please edit documentation in R/jira.R \name{make_jira_issue} \alias{make_jira_issue} -\title{Create JirAgileR Issue} +\title{Create JIRA Issue} \usage{ make_jira_issue( jira_domain_url, issue_key, + project_key, + summary, + description, issue_type, - status, resolution, - title, - description, + priority, + status, + labels, components, + affects_versions, + fix_versions, + assignee_name, creator_name, reporter_name, - assignee_name, comments = NULL ) } \arguments{ \item{jira_domain_url}{URL of JIRA domain (e.g. "https://project.org/jira")} -\item{issue_key}{issue key of JIRA issue (e.g. "PROJECT-68" or "GERONIMO-6723)} +\item{issue_key}{issue key of JIRA issue (e.g. "PROJECT-68" or "GERONIMO-6723")} + +\item{project_key}{key of the project that contains the JIRA issue (e.g. "SPARK" or "GERONIMO")} + +\item{summary}{summary of the issue (e.g. "Site Keeps Crashing")} + +\item{description}{more detailed description of issue (e.g. "The program keeps crashing because this reason")} \item{issue_type}{type of JIRA issue (e.g. "New Feature", "Task", "Bug")} +\item{resolution}{name of resolution for issue (e.g. "Fixed")} + +\item{priority}{the name of the priority of the issue (e.g. "Major", "Minor", "Trivial")} + \item{status}{status of issue for development (e.g. "In Progress")} -\item{resolution}{name of resolution for issue (e.g. "Fixed")} +\item{labels}{the labels of the project (e.g. "message", "mail", "jira")} -\item{title}{summary of the issue (e.g. "Site Keeps Crashing")} +\item{components}{list of components of issue (e.g. c("PS", "Tests"))} -\item{description}{more detailed description of issue (e.g. "The program keeps crashing because this reason")} +\item{affects_versions}{list of affected versions (e.g. c("3.1.6", "4.1.0"))} -\item{components}{components of issue separate by ; (e.g. "x-core;x-spring")} +\item{fix_versions}{list of fixed versions (e.g. c("3.1.5", "4.0.0"))} + +\item{assignee_name}{name of person the issue is being assigned to (e.g. "Joe Schmo")} \item{creator_name}{name of creator of issue (e.g. "John Doe")} \item{reporter_name}{name of reporter of issue (e.g. "Jane Doe")} -\item{assignee_name}{name of person the issue is being assigned to (e.g. "Joe Schmo")} - -\item{comments}{character vector where each element is a comment string (e.g. c("This is first comment", "This is second comment"))} +\item{comments}{character list where each element is a comment string (e.g. c("This is first comment", "This is second comment"))} } \value{ -A list which represents the JirAgileR JSON in memory +A list which represents the JIRA JSON in memory } \description{ Creates a single JIRA Issue as a list, which can be saved as a JSON with or without comments. -Note the JSON follows the format used by JirAgileR package when downloading -JSONs, instead of the format specified by JIRA. } \seealso{ Other {unittest}: diff --git a/man/parse_bugzilla_perceval_rest_issue_comments.Rd b/man/parse_bugzilla_perceval_rest_issue_comments.Rd index ee7e9702..d8788d60 100644 --- a/man/parse_bugzilla_perceval_rest_issue_comments.Rd +++ b/man/parse_bugzilla_perceval_rest_issue_comments.Rd @@ -32,6 +32,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_bugzilla_perceval_traditional_issue_comments.Rd b/man/parse_bugzilla_perceval_traditional_issue_comments.Rd index 230df1b1..06f9397d 100644 --- a/man/parse_bugzilla_perceval_traditional_issue_comments.Rd +++ b/man/parse_bugzilla_perceval_traditional_issue_comments.Rd @@ -32,6 +32,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_bugzilla_rest_comments.Rd b/man/parse_bugzilla_rest_comments.Rd index b6d295e6..8121d873 100644 --- a/man/parse_bugzilla_rest_comments.Rd +++ b/man/parse_bugzilla_rest_comments.Rd @@ -25,6 +25,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_bugzilla_rest_issues.Rd b/man/parse_bugzilla_rest_issues.Rd index cf217673..69d55e6b 100644 --- a/man/parse_bugzilla_rest_issues.Rd +++ b/man/parse_bugzilla_rest_issues.Rd @@ -27,6 +27,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_bugzilla_rest_issues_comments.Rd b/man/parse_bugzilla_rest_issues_comments.Rd index c5953688..68939e2c 100644 --- a/man/parse_bugzilla_rest_issues_comments.Rd +++ b/man/parse_bugzilla_rest_issues_comments.Rd @@ -29,6 +29,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_commit_message_id.Rd b/man/parse_commit_message_id.Rd index a081094f..1fe5fd3f 100644 --- a/man/parse_commit_message_id.Rd +++ b/man/parse_commit_message_id.Rd @@ -24,6 +24,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_dependencies.Rd b/man/parse_dependencies.Rd index 30c52d09..9cd9d487 100644 --- a/man/parse_dependencies.Rd +++ b/man/parse_dependencies.Rd @@ -33,6 +33,7 @@ Other parsers: \code{\link{parse_commit_message_id}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_dv8_clusters.Rd b/man/parse_dv8_clusters.Rd index e65265ef..474205be 100644 --- a/man/parse_dv8_clusters.Rd +++ b/man/parse_dv8_clusters.Rd @@ -22,6 +22,7 @@ Other parsers: \code{\link{parse_commit_message_id}()}, \code{\link{parse_dependencies}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_gitlog.Rd b/man/parse_gitlog.Rd index d8ee6999..5552e83c 100644 --- a/man/parse_gitlog.Rd +++ b/man/parse_gitlog.Rd @@ -28,6 +28,7 @@ Other parsers: \code{\link{parse_commit_message_id}()}, \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, diff --git a/man/parse_jira.Rd b/man/parse_jira.Rd index 2a303094..26b2da1f 100644 --- a/man/parse_jira.Rd +++ b/man/parse_jira.Rd @@ -2,19 +2,31 @@ % Please edit documentation in R/jira.R \name{parse_jira} \alias{parse_jira} -\title{Parse Jira issue and comments} +\title{Parse JIRA Issues and Comments} \usage{ -parse_jira(json_path) +parse_jira(json_folder_path) } \arguments{ -\item{json_path}{path to jira json (issues or issues with comments) obtained using `download_jira_data.Rmd`.} +\item{json_folder_path}{is a folder path containing a set of jira_issues as json files.} } \value{ A named list of two named elements ("issues", and "comments"), each containing a data.table. -Note the comments element will be empty if the downloaded json only contain issues. } \description{ -Parse Jira issue and comments +Parses JIRA issues without or with comments contained in a folder following a standardized file nomenclature. +as obtained from \code{\link{download_jira_issues}}. A named list with two elements (issues, comments) is returned +containing the issue table and optionally comments table. +} +\details{ +The following fields are expected on the raw data: + +issuekey, issuetype, components, creator, created, description, reporter, status, resolution +resolutiondate, assignee, updated, comment, priority, votes, watches, versions, fixVersions, labels + +which are the default parameters of \code{\link{download_jira_issues}}. If the `comment` field is +specified, then the comments table is included. + +If a field is not present in an issue, then its value will be NA. } \seealso{ Other parsers: @@ -27,6 +39,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_mbox}()}, \code{\link{parse_nvdfeed}()} diff --git a/man/parse_jira_latest_date.Rd b/man/parse_jira_latest_date.Rd new file mode 100644 index 00000000..f5b8b18f --- /dev/null +++ b/man/parse_jira_latest_date.Rd @@ -0,0 +1,39 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{parse_jira_latest_date} +\alias{parse_jira_latest_date} +\title{Parse JIRA current issue} +\usage{ +parse_jira_latest_date(json_folder_path) +} +\arguments{ +\item{json_folder_path}{path to save folder containing JIRA issue and/or comments json files.} +} +\value{ +The name of the jira issue file with the latest created date that was created/downloaded for +use by the Jira downloader refresher +} +\description{ +Returns the file containing the most current issue in the specified folder. +} +\details{ +The folder assumes the following convention: "(PROJECTKEY)_(uniextimestamp_lowerbound)_(unixtimestamp_upperbound).json" +For example: "SAILUH_1231234_2312413.json". This nomenclature is defined by \code{\link{download_jira_issues}}. +} +\seealso{ +Other parsers: +\code{\link{parse_bugzilla_perceval_rest_issue_comments}()}, +\code{\link{parse_bugzilla_perceval_traditional_issue_comments}()}, +\code{\link{parse_bugzilla_rest_comments}()}, +\code{\link{parse_bugzilla_rest_issues_comments}()}, +\code{\link{parse_bugzilla_rest_issues}()}, +\code{\link{parse_commit_message_id}()}, +\code{\link{parse_dependencies}()}, +\code{\link{parse_dv8_clusters}()}, +\code{\link{parse_gitlog}()}, +\code{\link{parse_jira_rss_xml}()}, +\code{\link{parse_jira}()}, +\code{\link{parse_mbox}()}, +\code{\link{parse_nvdfeed}()} +} +\concept{parsers} diff --git a/man/parse_jira_rss_xml.Rd b/man/parse_jira_rss_xml.Rd index 3cbb44d6..38bb6948 100644 --- a/man/parse_jira_rss_xml.Rd +++ b/man/parse_jira_rss_xml.Rd @@ -34,6 +34,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()}, \code{\link{parse_nvdfeed}()} diff --git a/man/parse_mbox.Rd b/man/parse_mbox.Rd index ba905a74..f048bd48 100644 --- a/man/parse_mbox.Rd +++ b/man/parse_mbox.Rd @@ -29,6 +29,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_nvdfeed}()} diff --git a/man/parse_nvdfeed.Rd b/man/parse_nvdfeed.Rd index bf2a6be8..e861f2a3 100644 --- a/man/parse_nvdfeed.Rd +++ b/man/parse_nvdfeed.Rd @@ -24,6 +24,7 @@ Other parsers: \code{\link{parse_dependencies}()}, \code{\link{parse_dv8_clusters}()}, \code{\link{parse_gitlog}()}, +\code{\link{parse_jira_latest_date}()}, \code{\link{parse_jira_rss_xml}()}, \code{\link{parse_jira}()}, \code{\link{parse_mbox}()} diff --git a/man/refresh_jira_issues.Rd b/man/refresh_jira_issues.Rd new file mode 100644 index 00000000..6e7118eb --- /dev/null +++ b/man/refresh_jira_issues.Rd @@ -0,0 +1,74 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/jira.R +\name{refresh_jira_issues} +\alias{refresh_jira_issues} +\title{Refresh JIRA Issues and/or Comments} +\usage{ +refresh_jira_issues( + domain, + jql_query, + fields, + username = NULL, + password = NULL, + save_folder_path, + max_results, + max_total_downloads, + verbose +) +} +\arguments{ +\item{domain}{Custom JIRA domain URL} + +\item{jql_query}{Specific query string to specify criteria for fetching} + +\item{fields}{List of fields that are downloaded in each issue} + +\item{username}{the JIRA username} + +\item{password}{the JIRA password/API token} + +\item{save_folder_path}{Path that files will be saved in .json format} + +\item{max_results}{(optional) the [maximum number of results](https://confluence.atlassian.com/jirakb/how-to-use-the-maxresults-api-parameter-for-jira-issue-search-rest-api-1251999998.html) +to download per page. Default is 50.} + +\item{max_total_downloads}{Maximum downloads per function call. +This parameter specifies how many issues should be downloaded. Subsequent API calls will stop if they +reach or surpass this value.} + +\item{verbose}{Set verbose=TRUE to print execution details.} +} +\description{ +Uses the adopted file name convention by \code{\link{download_jira_issues}} to identify +the latest downloaded JIRA issue key KEY-i, and calls +\code{\link{download_jira_issues_by_issue_key}} with lower bound KEY-(i+1) to download all +newer issues. +} +\details{ +If the directory is empty, then all issues will be downloaded. This function can therefore +be used in the specified folder to continuously refresh available issues and/or comments +data. +} +\seealso{ +\code{\link{download_jira_issues_by_date}} to download JIRA issues and/or comments by date range + +\code{\link{download_jira_issues_by_issue_key}} to download JIRA issues and/or comments by issue key range + +\code{\link{download_jira_issues}} for more flexibility in specifying the JQL query + +\code{\link{parse_jira}} to parse jira data with or without comments + +\code{\link{parse_jira_latest_date}} to retrieve the file path of the latest issue key + +Other downloaders: +\code{\link{download_jira_issues_by_date}()}, +\code{\link{download_jira_issues_by_issue_key}()}, +\code{\link{download_jira_issues}()} + +Other jira: +\code{\link{download_jira_issues_by_date}()}, +\code{\link{download_jira_issues_by_issue_key}()}, +\code{\link{download_jira_issues}()} +} +\concept{downloaders} +\concept{jira} diff --git a/tests/testthat/test-jira.R b/tests/testthat/test-jira.R index aa0f52fb..9229c763 100644 --- a/tests/testthat/test-jira.R +++ b/tests/testthat/test-jira.R @@ -7,35 +7,35 @@ test_that("Incorrect jira issue comments path fails parse_jira", { test_that("parse_jira parses one issue with two components as one row", { - jira_json_path <- example_jira_issue_components(folder_path = "/tmp", - folder_name = "issue_with_components") + jira_json_path <- example_jira_issue_components(folder_path = "/tmp", folder_name = "single_issue") - issues_comments_list <- parse_jira(json_path = jira_json_path) + issues_comments_list <- parse_jira(json_folder_path = jira_json_path) issues <- issues_comments_list[["issues"]] - io_delete_folder(folder_path="/tmp", folder_name="issue_with_components") + io_delete_folder(folder_path="/tmp", folder_name="single_issue") expect_equal(nrow(issues),1) -}) + } +) test_that("parse_jira parses two issues as two rows", { - jira_json_path <- example_jira_two_issues(folder_path = "/tmp", - folder_name = "one_issue_two_comments") - issues_comments_list <- parse_jira(json_path = jira_json_path) + jira_json_path <- example_jira_two_issues(folder_path = "/tmp", folder_name = "two_issues") + issues_comments_list <- parse_jira(json_folder_path = jira_json_path) issues <- issues_comments_list[["issues"]] - io_delete_folder(folder_path="/tmp", folder_name="one_issue_two_comments") + io_delete_folder(folder_path="/tmp", folder_name="two_issues") expect_equal(nrow(issues),2) -}) + } +) test_that("parse_jira parses one issue with two comments as two rows", { - jira_json_path <- example_jira_issue_comments(folder_path = "/tmp", - folder_name = "one_issue_two_comments") - issues_comments_list <- parse_jira(json_path = jira_json_path) + jira_json_path <- example_jira_issue_comments(folder_path = "/tmp", folder_name = "one_issue_two_comments") + issues_comments_list <- parse_jira(json_folder_path = jira_json_path) comments <- issues_comments_list[["comments"]] io_delete_folder(folder_path="/tmp", folder_name="one_issue_two_comments") expect_equal(nrow(comments),2) -}) + } +) diff --git a/vignettes/download_jira_data.Rmd b/vignettes/download_jira_data.Rmd deleted file mode 100644 index 07050339..00000000 --- a/vignettes/download_jira_data.Rmd +++ /dev/null @@ -1,136 +0,0 @@ ---- -title: "Download JIRA Issues and Comments" -output: - html_document: - toc: true - number_sections: true -vignette: > - %\VignetteEngine{knitr::rmarkdown} - %\VignetteIndexEntry{Download JIRA Issues and Commentss} - %\VignetteEncoding{UTF-8} ---- - - -```{r} -rm(list = ls()) -seed <- 1 -set.seed(seed) -``` - -```{r warning=FALSE,message=FALSE} -require(kaiaulu) -require(data.table) -require(JirAgileR) -require(knitr, quietly = T) -require(dplyr, quietly = T) -require(jsonlite) -``` - -# Introduction - -This example is adapted from the JirAgileR package [README.md](https://github.com/matbmeijer/JirAgileR). - -As usual, the first step is to load the project configuration file. - -# Project Configuration File - -In this notebook, we will obtain data from the issue tracker JIRA. We will use [Apache's Geronimo open source project](https://geronimo.apache.org). Refer to the `conf/` folder on Kaiaulu's git repository for Geronimo and other project configuration files. It is in this project configuration file we specify where Kaiaulu can find the jira sources from Geronimo. We will use the issue_tracker -> jira fields only. In regards to the "issues" and "issue_comments" fields, these should be set to paths where you want to store the jira data. Then, you can access this jira data later using these same paths. - -```{r} -conf <- yaml::read_yaml("../conf/geronimo.yml") -issue_tracker_domain <- conf[["issue_tracker"]][["jira"]][["domain"]] -issue_tracker_project_key <- conf[["issue_tracker"]][["jira"]][["project_key"]] -save_path_issue_tracker_issues <- conf[["issue_tracker"]][["jira"]][["issues"]] -save_path_issue_tracker_issue_comments <- conf[["issue_tracker"]][["jira"]][["issue_comments"]] - -``` - -# Specifying the JIRA credentials - -In the project configuration file, we define the domain which JIRA is hosted. The example configuration file uses the domain for Apache Software Foundation. We can use this information to queue the domain to identify all project keys available in its JIRA: - -```{r eval = FALSE} -# Save credentials to pass them only one time -save_jira_credentials(domain = issue_tracker_domain) - -# Get full list of projects in domain -apache_jira_projects <- data.table(get_jira_projects()) #%>% - #select(key, name) - -``` - -You can load `apache_jira_projects` for the list of projects available. However, this step is optional. - -# Download Issues (Without Comments) - -In the project configuration file, we also specify the project configuration key (you can also type it directly for the function call). Here, we used the key `GERONIMO`. You can also manually explore the existing issues from a given project by visiting it [on your browser](https://issues.apache.org/jira/projects/GERONIMO/issues/GERONIMO-734?filter=allopenissues). Please see the function documentation warning for `get_jira_issues` on respecting the number of function calls to the server to avoid being IP blocked. - -The following function will download all the available issues. For more details, see the associated package documentation. This data can be used to obtain the `bug_count` metric. See the associated R notebook `bug_count.Rmd` for details. - -We will download the data on `verbose`, and as json (`as.data.frame = FALSE`), as the turning the later appears to cause column names to not match values, ignores the requested fields, and gives inconsistent column names. Kaiaulu implements its own parser for the json file, so you will still be able to obtain a table in the end without the associated issues. - -```{r eval = FALSE} -# Retrieve the issues from a single project - in this case the project QTWB from bugreports.qt.io. See documentation to define which fields to see -json_issues <- get_jira_issues(jql_query = paste0("project='",issue_tracker_project_key,"'"), - fields = c("summary", - "description", - "creator", - "assignee", - "reporter", - "issuetype", - "status", - "resolution", - "components", - "created", - "updated", - "resolutiondate"), - verbose=TRUE, - as.data.frame = FALSE) - -jsonlite::write_json(json_issues,save_path_issue_tracker_issues) -``` - -The json will be downloaded on the path specified in the project configuration file, which by default is `kaiaulu/kaiaulu/rawdata/issue_tracker`. We can then use Kaiaulu's function to parse the data into a tabular format. Since our request did not include the `comment` field, only the issues table will be available. A few rows of the json issues is shown next: - -```{r} -jira_issues_list <- parse_jira(save_path_issue_tracker_issues) -jira_issues <- jira_issues_list[["issues"]] -jira_comments <- jira_issues_list[["comments"]] -kable(jira_issues[7:8]) -``` - -# Download Issue with Comments - -In the same manner as before, we can perform the same function call, but including the field `comment`. This will result in the same table being generated but with the additional comment information per issue (if an issue has more than one comment, the issue id is repeated for each different comment). The comment is shown on the column `comment_comments_id`. - -The data of this table can be used to calculate `social smell metrics`, as it represents a form of developer communication. A notebook discussing how to use JIRA data as communication network and/or combining to mailing list data will be made available in the future. - -```{r eval = FALSE} -json_issue_comments <- get_jira_issues(jql_query = paste0("project='",issue_tracker_project_key,"'"), - fields = c("summary", - "description", - "creator", - "assignee", - "reporter", - "issuetype", - "status", - "resolution", - "components", - "created", - "updated", - "resolutiondate", - "comment"), - verbose=TRUE, - as.data.frame = FALSE) -jsonlite::write_json(json_issue_comments,save_path_issue_tracker_issue_comments) -``` - -Since this time around we requested the issue data and comments, when using the `parse_jira` function, both the issues and comments table will be available from the parser. Since the issue table was already displayed, the following show a few rows of the issue comments table: - -```{r} -jira_issue_comments <- parse_jira(save_path_issue_tracker_issue_comments) -jira_issues <- jira_issue_comments[["issues"]] -jira_comments <- jira_issue_comments[["comments"]] - -kable(jira_comments[55:56]) -``` diff --git a/vignettes/download_jira_issues.Rmd b/vignettes/download_jira_issues.Rmd new file mode 100644 index 00000000..40de0878 --- /dev/null +++ b/vignettes/download_jira_issues.Rmd @@ -0,0 +1,294 @@ +--- +title: "Download JIRA Issues and Comments" +output: + html_document: + toc: true + number_sections: true +vignette: > + %\VignetteEngine{knitr::rmarkdown} + %\VignetteIndexEntry{Download JIRA Issues and Comments} + %\VignetteEncoding{UTF-8} +--- + + +```{r} +rm(list = ls()) +seed <- 1 +set.seed(seed) +``` + +```{r warning=FALSE,message=FALSE} +require(kaiaulu) +require(data.table) +require(knitr, quietly = T) +require(jsonlite) +require(gt) +``` + +# Introduction + +This Notebook showcases how to obtain JIRA issues and comments using Kaiaulu with and without authentication. We also demonstrate how you can use the `refresher` capability: I.e. by calling the function again, Kaiaulu will only download more recent data in a folder, provided this folder issues were also obtained with the refresh function. This is useful for server-side deployment, in combination with CRON jobs, but can also be used locally to just get more recent data conveniently. + +While you have flexibility in choosing what type of fields should be included on the downloader JSON data, bear in mind only the default fields specified on the downloaders are available on the parser. These, however, are easily extensible. + +## Project Configuration File and Setup + +In this notebook, we will obtain data from the JIRA issue tracker. We will demonstrate it on Kaiaulu [JIRA sandbox](https://sailuh.atlassian.net/jira/). Refer to the `conf/` folder on Kaiaulu's git repository for Geronimo and other project configuration files. Using a different project configuration file should only require changing the path for the configuration, provided they do not require different authentication files. + +You should also have the folder structure created on your computer manually. Specifically, in the "issues" and "issue_comments" fields, you should specify the folder path where the data will be saved. We recommend you adopt the convention adopted in Kaiaulu if you are collaborating with others, as other sections of the configuration file used in other Notebooks will also adopt this overall structure. + +In general, projects such as Apache Software Foundation do not require authentication to obtain issue data. Authentication is only required to obtain more sensitive data. If you are using the Kaiaulu tool for an internal JIRA account, however, authentication may be useful for you. The free version of JIRA (e.g. https://sailuh.atlassian.net/) used in this Notebook also requires authentication. In the case of the free JIRA version, your API username is your account e-mail, and your password is an API token [you create](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/). We explain how to load your username and password below. + +To try out this Notebook, try the geronimo configuration file, and follow along the instructions below on how to remove the authentication parameters required for Kaiaulu. + + +First, we will load the Kaiaulu configuration file: + +```{r} +conf <- yaml::read_yaml("../conf/kaiaulu.yml") +# Project domain +issue_tracker_domain <- conf[["issue_tracker"]][["jira"]][["domain"]] +# Project key +issue_tracker_project_key <- conf[["issue_tracker"]][["jira"]][["project_key"]] +# Altered save paths. Important for naming conventions +save_path_issue_tracker_issues <- conf[["issue_tracker"]][["jira"]][["issues"]] +save_path_issue_tracker_issue_comments <- conf[["issue_tracker"]][["jira"]][["issue_comments"]] +# Unaltered save paths from config file for use with refresh function +refresh_issues <- conf[["issue_tracker"]][["jira"]][["issues"]] +``` + +If authentication is needed, save your username (e-mail) and password (API token) in a file, e.g. atlassian_credentials, where the first line is the username, and the second the API token, e.g. + +``` +jondoe@jondoe.com +jondoespassword +``` + +And remove the `eval = FALSE` from the code block below (note you do not want to use this code block if you are accessing a JIRA instance that you do not have an account such as Apache Software Foundation, or you will have authentication errors): + +```{r eval = FALSE} +if(file.exists("~/.ssh/atlassian_credentials")){ + credentials <- scan("~/.ssh/atlassian_credentials", what = "character", quiet = TRUE) + username <- credentials[1] + password <- credentials[2] +} +``` + +## Create the directories specified in the config files + +Before proceeding, make sure you created the folder structure you specified on the project configuration file. This Notebook will not create the folders automatically. + +# Downloading Issues + +Kaiaulu offers three ways to download issues with or without comment: By Date, by Issue Key, and Custom. The Date and Issue Key range functions wrap around the Custom function, serving as examples on how to customize it. + +## Issues by Date Range + +Using the `download_jira_issues_by_date()`, you can download issues based on their ['created'](https://support.atlassian.com/jira-software-cloud/docs/jql-fields/#Created) dates. Th acceptable formats are: + + * "yyyy/MM/dd HH:mm" + * "yyyy-MM-dd HH:mm" + * "yyyy/MM/dd" + * "yyyy-MM-dd" + +If both date range parameters are set to `NULL`, then all issues will be retrieved. Alternatively, if for example `date_lower_bound` is set to "2000-01-01" and `date_upper_bound` is set to "2005-01-01", then only issues created between these two dates will be retrieved. + +If you would like to retrieve issues only **after** a certain date, set `date_upper_bound=NULL` and `date_lower_bound` to the date. + +If you would like to retrieve only issues **before** a certain date, set `date_lower_bound=NULL` and date_upper_bound to the date. + +Note in the subsequent code block we specified the fields from the issue we are interested in downloading. Again, you may specify [any fields](https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issue-fields/#api-rest-api-2-field-get) of interest for the downloader, including custom fields of your JIRA instance, but the parser currently only support those listed here (in addition to comments data, not exemplified yet). + +Beware that even if only 3 issues exist in a JIRA, a large time range will still request several API calls (in contrast to the issue endpoint below). Therefore, it is advisable to use the issue key query instead which is explained in the next sub-section. + +```{r eval = FALSE} +# e.g. date_lower_bound <- "1970/01/01". +date_lower_bound <- "2023/11/16 21:00" +date_upper_bound <- "2023/11/17 21:00" + +all_issues <- download_jira_issues_by_date(issue_tracker_domain, + jql_query = paste0("project='",issue_tracker_project_key,"'"), + fields = c("summary", + "description", + "creator", + "assignee", + "reporter", + "issuetype", + "status", + "resolution", + "components", + "created", + "updated", + "resolutiondate", + "priority", + "votes", + "watches", + "versions", + "fixVersions", + "labels"), + username = username, + password = password, + save_folder_path = save_path_issue_tracker_issues, + max_results = 50, + max_total_downloads = 60, + date_lower_bound = date_lower_bound, + date_upper_bound = date_upper_bound, + verbose = TRUE) + +``` + +Note if authentication is not required, you should also comment the `username` and `password` parameters, or the information will be used to authenticate a JIRA instance you do not have an account, resulting in authorization errors. The downloaded data can also parsed in Kaiaulu: + +```{r} +parsed_jira_issues <- parse_jira(save_path_issue_tracker_issues) + +head(parsed_jira_issues[["issues"]]) %>% + gt(auto_align = FALSE) +``` + +## Issues by Key Range + +A way to download issues requiring less API calls is using the issue keys, using `download_jira_issues_by_issue_key()`. + +Instead of specifying the date range, you can instead specify the issue range using `issue_key_lower_bound` and `issue_key_upper_bound` to retrieve only JIRA issues with the issue key between these bounds. These parameters need to be specified in accordance with the issue key format relevant to your project. The default format is -. An example is if issue_key_lower_bound is set to "GERONIMO-740" and issue_key_upper_bound is set to "GERONIMO-6000", then only issues with issue keys between these two issue keys will be retrieved. + +Some special cases are also possible: If both lower and upper bound are set to NULL, then all issues will be retrieved as the request will not be bounded. + +If issues with issue key greater than an issue key value are desired, set issue_key_upper_bound to NULL and issue_key_lower_bound to the issue key value. + +If issues with issue key less than an issue key value are desired, set issue_key_upper_bound to the issue key and issue_key_lower_bound to NULL. + +In the subsequent codeblock, note we also include a new field, `comment`. This inclusion makes it so comments data is also downloaded. The `comment` field can also be included on any downloader or discussed in this Notebook. We also use a different folder path here, `save_folder_path = save_path_issue_tracker_issue_comments`, which you should have specified on your project configuration file. We separate issues with and without comments for simplicity and clarity into separate folders. + +```{r eval = FALSE} +# eg issueKey_lower_bound <- "GERONIMO-740" +#issue_key_lower_bound <- "GERONIMO-500" +#issue_key_upper_bound <- "GERONIMO-560" + +issue_key_lower_bound <- "SAILUH-1" +issue_key_upper_bound <- "SAILUH-3" + +all_issues <- download_jira_issues_by_issue_key(domain = issue_tracker_domain, + jql_query = paste0("project='",issue_tracker_project_key,"'"), + fields = c("summary", + "description", + "creator", + "assignee", + "reporter", + "issuetype", + "status", + "resolution", + "components", + "created", + "updated", + "resolutiondate", + "priority", + "votes", + "watches", + "versions", + "fixVersions", + "labels", + "comment"), + username = username, + password = password, + save_folder_path = save_path_issue_tracker_issue_comments, + max_results = 50, + max_total_downloads = 60, + issue_key_lower_bound = issue_key_lower_bound, + issue_key_upper_bound = issue_key_upper_bound, + verbose = TRUE) +``` + + +```{r} +names(parsed_jira_issues) +``` + +Observe the parser function returns a named list of two tables. We showed before how to access the issues information when we downloaded data by date range. Here, we show the comments table. + +```{r} +parsed_jira_issues <- parse_jira(save_path_issue_tracker_issue_comments) + + +head(parsed_jira_issues[["comments"]]) %>% + gt(auto_align = FALSE) +``` + +## Downloading Issues with Customizations + +If you require more flexibility when using the downloader, you can use the `download_jira_issues`. This function does not append the time range or date range when requesting the data. You can study the previous two functions code to see how they use `download_jira_issues` to query specific date ranges and time ranges. For details, see the function documentation. + +```{r eval=FALSE} +all_issues <- download_jira_issues(issue_tracker_domain, + credentials, + jql_query = paste0("project='",issue_tracker_project_key,"'"), + fields = c("summary", + "description", + "creator", + "assignee", + "reporter", + "issuetype", + "status", + "resolution", + "components", + "created", + "updated", + "resolutiondate", + "priority", + "votes", + "watches", + "versions", + "fixVersions", + "labels"), + username = username, + password = password, + save_folder_path = save_path_issue_tracker_issue_comments, + max_results = 50, + max_total_downloads = 5000, + search_query = NULL, + verbose = TRUE) +``` + + +# Refresh Issue and Comments Data + +There are a few instances in which downloading the issue data with comments does not capture all the issues: + + 1. The JIRA rest API rate limit (currently 5000/hour) is reached. + 2. Function ends before completing. + 3. The `max_downloads` parameter was set to a value lower than total issues. + 4. More issues were since posted after downloading. + +Given a folder using any of the downloaders in this Notebook (with or without comments), you can use the `refresh_jira_issues` function to add additional issue data since the last issue was downloaded. This function relies on the naming convention the downloaders utilize on the file to perform this operation. For details, see the function documentation. + +```{r eval = FALSE} +refresh_jira_issues(issue_tracker_domain, + credentials, + jql_query = paste0("project='",issue_tracker_project_key,"'"), + fields = c("summary", + "description", + "creator", + "assignee", + "reporter", + "issuetype", + "status", + "resolution", + "components", + "created", + "updated", + "resolutiondate", + "priority", + "votes", + "watches", + "versions", + "fixVersions", + "labels", + "comment"), + save_path_issue_tracker_issues, + max_results = 50, + max_total_downloads = 5000, + refresh_issues, + verbose = TRUE) + +``` +