{"id":46054,"date":"2022-08-15T15:12:42","date_gmt":"2022-08-15T15:12:42","guid":{"rendered":"https:\/\/harchi90.com\/machine-learning-concluded-did-the-no-code-tools-beat-manual-analysis\/"},"modified":"2022-08-15T15:12:42","modified_gmt":"2022-08-15T15:12:42","slug":"machine-learning-concluded-did-the-no-code-tools-beat-manual-analysis","status":"publish","type":"post","link":"https:\/\/harchi90.com\/machine-learning-concluded-did-the-no-code-tools-beat-manual-analysis\/","title":{"rendered":"Machine learning, concluded: Did the \u201cno-code\u201d tools beat manual analysis?"},"content":{"rendered":"<div itemprop=\"articleBody\">\n<figure class=\"intro-image intro-left\">\n  <figcaption class=\"caption\">\n<p>Aurich Lawson |  Getty Images<\/p>\n<\/figcaption><\/figure>\n<aside id=\"social-left\" class=\"social-left\" aria-label=\"Read the comments or share this article\">\n<\/aside>\n<p><!-- cache hit 546:single\/related:50c72e245a0fe242c0f106aae3eef951 --><!-- empty --><\/p>\n<p>I am not a data scientist.  And while I know my way around a Jupyter notebook and have written a good amount of Python code, I do not profess to be anything close to a machine learning expert.  So when I performed the first part of our no-code\/low-code machine learning experiment and got better than a 90 percent accuracy rate on a model, I doubt I had done something wrong.<\/p>\n<p>If you haven&#8217;t been following along thus far, here&#8217;s a quick review before I direct you back to the first two articles in this series.  To see how much machine learning tools for the rest of us had advanced\u2014and to redeem myself for the unwinnable task I had been assigned with machine learning last year\u2014I took a well-worn heart attack data set from an archive at the University of California-Irvine and tried to outperform data science students&#8217; results using the &#8220;easy button&#8221; of Amazon Web Services&#8217; low-code and no-code tools.<\/p>\n<p>The whole point of this experiment was to see:<\/p>\n<ul>\n<li>Whether a relative novice could use these tools and accurately<\/li>\n<li>Whether the tools were more cost-effective than finding someone who knew what the heck they were doing and handing it off to them<\/li>\n<\/ul>\n<p>That&#8217;s not exactly a true picture of how machine learning projects usually happen.  And as I found, the &#8220;no-code&#8221; option that Amazon Web Services provides\u2014SageMaker Canvas\u2014is intended to work hand-in-hand with the more data science-y approach of SageMaker Studio.  But Canvas outperformed what I was able to do with the low-code approach of Studio\u2014though probably because of my less-than-skilled data-handling hands.<\/p>\n<p>(For those who have not read the previous two articles, now is the time to catch up: Here&#8217;s part one, and here&#8217;s part two.)                                            <\/p>\n<aside class=\"ad_wrapper\" aria-label=\"In Content advertisement\">\n    <span class=\"ad_notice\">Advertisement <\/span>    <\/p>\n<\/aside>\n<h2>Assessing the robot&#8217;s work<\/h2>\n<p>Canvas allowed me to export a sharable link that opened the model I created with my full build from the 590-plus rows of patient data from the Cleveland Clinic and the Hungarian Institute of Cardiology.  That link gave me a little more insight into what went on inside Canvas&#8217; very black box with Studio, a Jupyter-based platform for doing data science and machine learning experiments.<\/p>\n<p>As its name slyly suggests, Jupyter is based on Python.  It is a web-based interface to a container environment that allows you to spin up kernels based on different Python implementations, depending on the task.<\/p>\n<figure class=\"image shortcode-img center full\" style=\"width:100%\"><img loading=\"lazy\" alt=\"Examples of the different kernel containers available in Studio.\" src=\"https:\/\/i0.wp.com\/cdn.arstechnica.net\/wp-content\/uploads\/2022\/08\/sean-aiml3-01.png?resize=730%2C595&#038;ssl=1\" width=\"730\" height=\"595\" data-recalc-dims=\"1\" \/><figcaption class=\"caption\">\n<p>Examples of the different kernel containers available in Studio.<\/p>\n<\/figcaption><\/figure>\n<p>Kernels can be populated with whatever modules the project requires when you&#8217;re doing code-focused explorations, such as the Python Data Analysis Library (pandas) and SciKit-Learn (sklearn).  I used a local version of Jupyter Lab to do most of my initial data analysis to save on AWS compute time.<\/p>\n<p>The Studio environment created with the Canvas link included some pre-built content providing insight into the model Canvas produced\u2014some of which I discussed briefly in the last article:<\/p>\n<figure class=\"image shortcode-img full-width\" style=\"width:980px\"><img loading=\"lazy\" alt=\"Model details from the Canvas best-of-show in Studio.\" src=\"https:\/\/i0.wp.com\/cdn.arstechnica.net\/wp-content\/uploads\/2022\/08\/sean-aiml3-02-980x445.png?resize=980%2C445&#038;ssl=1\" width=\"980\" height=\"445\" data-recalc-dims=\"1\" \/><figcaption class=\"caption\">\n<div class=\"caption-text\">enlarge <span class=\"sep\">\/<\/span> Model details from the Canvas best-of-show in Studio.<\/div>\n<\/figcaption><\/figure>\n<p>Some of the details included the hyperparameters used by the best-tuned version of the model created by Canvas:<\/p>\n<figure class=\"image shortcode-img full-width\" style=\"width:980px\"><img loading=\"lazy\" alt=\"Model hyperparameters.\" src=\"https:\/\/i0.wp.com\/cdn.arstechnica.net\/wp-content\/uploads\/2022\/08\/sean-aiml3-03-980x355.png?resize=980%2C355&#038;ssl=1\" width=\"980\" height=\"355\" data-recalc-dims=\"1\" \/><figcaption class=\"caption\">\n<div class=\"caption-text\">enlarge <span class=\"sep\">\/<\/span> Model hyperparameters.<\/div>\n<\/figcaption><\/figure>\n<p>Hyperparameters are tweaks that AutoML made to calculations by the algorithm to improve the accuracy, as well as some basic housekeeping\u2014the SageMaker instance parameters, the tuning metric (&#8220;F1,&#8221; which we&#8217;ll discuss in a moment), and other inputs.  These are all pretty standard for a binary classification like ours.<\/p>\n<p>The model overview in Studio provided some basic information about the model produced by Canvas, including the algorithm used (XGBoost) and the relative importance of each of the columns rated with something called SHAP values.  SHAP is a really horrible acronym that stands for &#8220;SHApley Additive exPlanations,&#8221; which is a game theory-based method of extracting each data feature&#8217;s contribution to a change in the model output.  It turns out that &#8220;maximum heart rate achieved&#8221; had negligible impact on the model, while thalassemia (&#8220;thall&#8221;) and angiogram results (&#8220;caa&#8221;)\u2014data points we had significant missing data for\u2014had more impact than I wanted them to .  I couldn&#8217;t just drop them, apparently.  So I downloaded a performance report for the model to get more detailed information on how the model held up:<\/p>\n<figure class=\"image shortcode-img center full\" style=\"width:100%\"><img loading=\"lazy\" src=\"https:\/\/i0.wp.com\/cdn.arstechnica.net\/wp-content\/uploads\/2022\/08\/sean-aiml3-04.png?resize=651%2C449&#038;ssl=1\" width=\"651\" height=\"449\" data-recalc-dims=\"1\" \/><\/figure>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Aurich Lawson | Getty Images I am not a data scientist. And while I know my way around a Jupyter notebook and have written a good amount of Python code, I do not profess to be anything close to a machine learning expert. So when I performed the first part of our no-code\/low-code machine learning &hellip;<\/p>\n<p class=\"read-more\"> <a class=\"\" href=\"https:\/\/harchi90.com\/machine-learning-concluded-did-the-no-code-tools-beat-manual-analysis\/\"> <span class=\"screen-reader-text\">Machine learning, concluded: Did the \u201cno-code\u201d tools beat manual analysis?<\/span> Read More &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"default","ast-global-header-display":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[4],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":40239,"url":"https:\/\/harchi90.com\/setting-our-heart-attack-predicting-ai-loose-with-no-code-tools\/","url_meta":{"origin":46054,"position":0},"title":"Setting our heart-attack-predicting AI loose with \u201cno-code\u201d tools","date":"August 9, 2022","format":false,"excerpt":"enlarge \/ Ahhh, the easy button!Aurich Lawson | Getty Images This is the second episode in our exploration of \"no-code\" machine learning. In our first article, we laid out our problem set and discussed the data we would use to test whether a highly automated ML tool designed for business\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":22007,"url":"https:\/\/harchi90.com\/tiny-pinball-machine-also-runs-x86-code\/","url_meta":{"origin":46054,"position":1},"title":"Tiny Pinball Machine Also Runs X86 Code","date":"July 22, 2022","format":false,"excerpt":"As arcades become more and more rare, plenty of pinball enthusiasts are moving these intricate machines to their home collections in basements, garages, and guest rooms. But if you're not fortunate enough to live in a home that can support a space-intensive hobby like pinball machines, there are some solutions\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":31904,"url":"https:\/\/harchi90.com\/linus-torvalds-looses-linux-5-19-using-asahi-linux-on-a-mac-the-register\/","url_meta":{"origin":46054,"position":2},"title":"Linus Torvalds looses Linux 5.19 using Asahi Linux on a Mac \u2022 The Register","date":"August 1, 2022","format":false,"excerpt":"Linus Torvalds has released version 5.19 of the project, and hailed Apple's homebrew silicon \u2013 and the Asahi Linux distribution that runs on it \u2013 for making Arm-powered computers useful for developers. In his announcement of the release, Torvalds called out work to support for the made-in-China Loongarch RISC architecture\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":47277,"url":"https:\/\/harchi90.com\/the-new-usb-rubber-ducky-is-more-dangerous-than-ever\/","url_meta":{"origin":46054,"position":3},"title":"The new USB Rubber Ducky is more dangerous than ever","date":"August 16, 2022","format":false,"excerpt":"The USB Rubber Ducky is back with a vengeance. The much-loved hacking tool has a new incarnation, released to coincide with the Def Con hacking conference this year, and creator Darren Kitchen was on hand to explain it to The Verge. We tested out some of the new features and\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":9096,"url":"https:\/\/harchi90.com\/microsoft-releases-visual-studio-code-server-preview-the-register\/","url_meta":{"origin":46054,"position":4},"title":"Microsoft releases Visual Studio Code Server preview \u2022 The Register","date":"July 9, 2022","format":false,"excerpt":"Microsoft has tossed its line into the increasingly crowded ring of code-anywhere developer tools with a private preview of Visual Studio Code Server. The product follows the Remote Development extensions for Visual Studio Code, which permitted apps to be developed using a local copy of the IDE and a \"remote\"\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":40787,"url":"https:\/\/harchi90.com\/pixel-7-ultra-is-making-headlines-is-questionable-at-best\/","url_meta":{"origin":46054,"position":5},"title":"Pixel 7 Ultra is making headlines, is questionable at best","date":"August 10, 2022","format":false,"excerpt":"Well, it's August and that means it is time for the Pixel hype\/rumor train to really start revving up. I talked a few months ago about the fact that Google's relatively-transparent unveiling of the Pixel 7 and Pixel 7 Pro at Google I\/O 2022 took a lot of the steam\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chromeunboxed.com\/wp-content\/uploads\/2022\/05\/Pixel-7-and-Pixel-7-Pro.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]}],"fifu_image_url":"https:\/\/cdn.arstechnica.net\/wp-content\/uploads\/2022\/08\/robot-holding-monitored-heart-760x380.jpg","_links":{"self":[{"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/posts\/46054"}],"collection":[{"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/comments?post=46054"}],"version-history":[{"count":0,"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/posts\/46054\/revisions"}],"wp:attachment":[{"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/media?parent=46054"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/categories?post=46054"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/harchi90.com\/wp-json\/wp\/v2\/tags?post=46054"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}