{"id":40,"date":"2022-02-08T20:12:50","date_gmt":"2022-02-08T20:12:50","guid":{"rendered":"https:\/\/blogs.oregonstate.edu\/kblack\/?p=40"},"modified":"2022-02-08T20:12:50","modified_gmt":"2022-02-08T20:12:50","slug":"automating-workflows","status":"publish","type":"post","link":"https:\/\/blogs.oregonstate.edu\/kblack\/2022\/02\/08\/automating-workflows\/","title":{"rendered":"Automating Workflows"},"content":{"rendered":"\n<p>In my internship, I\u2019m starting a project in which the goal is to keep a record of files that store variables used in a system. Currently, the place where these files are stored does not maintain any type of changelog. User changes to these files are not recorded. My project develops a process to monitor and record file changes by utilizing GitHub for version control, Jenkins Pipeline for automation, and Snap Logic to build the files. Since a lot of this is new for me I thought it would be great to run through these technologies and how they could be used to maintain an automated system to monitor, record, and implement file changes. \u00a0<\/p>\n\n\n\n<p>Most of us are familiar with using Git and GitHub for version control. Oftentimes remote repositories contain a \u2018main\u2019 branch that contains the reviewed and approved code. Source code changes made by users are housed in branches that are merged with the main branch upon approval. Depending on how the repository is set up, a user submitting changes must submit a pull request (PR). Within the pull request various workflow actions, as well as a code review(s), take place. Workflow actions could check for formatting, linting, etc\u2026<\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/www.jenkins.io\/doc\/\" target=\"_blank\">Jenkins<\/a> is a CI\/CD automation tool for building, testing, delivering, or deploying software. In Jenkins, a <a rel=\"noreferrer noopener\" href=\"https:\/\/www.jenkins.io\/doc\/book\/pipeline\/\" target=\"_blank\">Pipeline<\/a> can be used to get software or code from a version control system like GitHub to an end-user. A Pipeline is built using a Jenkinsfile which can be written using declarative or scripted syntax. Jenkins declarative syntax is written using <a rel=\"noreferrer noopener\" href=\"https:\/\/groovy-lang.org\/documentation.html\" target=\"_blank\">Groovy<\/a>, an object-oriented language for the Java platform. Jenkins can be integrated with GitHub so that a Jenkins Pipeline is triggered when a user merges a PR to a particular branch.<\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/www.snaplogic.com\/\" target=\"_blank\">SnapLogic<\/a> is an Integration Platform as a Service (IPaaS) tool to connect data sources and automate workflows, like ETL (extract, transform, load) flows, through the use of pipelines. SnapLogic can connect and retrieve data from a variety of sources, transform the data, then write files or load data to end-user systems.<\/p>\n\n\n\n<p>With regards to my project, the above-mentioned files will be housed in a GitHub repository so that a record of changes to files can be maintained. When a user wants to make changes to the files, they will submit a PR to merge their changes to main. Once the PR is approved, a Jenkins Pipeline will determine what files are newly added, modified, and\/or deleted. These file changes will be mapped to a Groovy object and ultimately output as a JSON string to SnapLogic using the HTTP post method. SnapLogic pipelines will verify the data, then write or delete the files on the end-user. <\/p>\n\n\n\n<p>Just a snapshot of some new technologies I&#8217;m working with and how they aid in automation. <\/p>\n","protected":false},"excerpt":{"rendered":"<p>In my internship, I\u2019m starting a project in which the goal is to keep a record of files that store variables used in a system. Currently, the place where these files are stored does not maintain any type of changelog. User changes to these files are not recorded. My project develops a process to monitor [&hellip;]<\/p>\n","protected":false},"author":11980,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-40","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/posts\/40","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/users\/11980"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/comments?post=40"}],"version-history":[{"count":6,"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/posts\/40\/revisions"}],"predecessor-version":[{"id":47,"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/posts\/40\/revisions\/47"}],"wp:attachment":[{"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/media?parent=40"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/categories?post=40"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/kblack\/wp-json\/wp\/v2\/tags?post=40"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}