{"id":862,"date":"2024-12-11T11:08:14","date_gmt":"2024-12-11T11:08:14","guid":{"rendered":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/?p=862"},"modified":"2024-12-11T11:17:33","modified_gmt":"2024-12-11T11:17:33","slug":"why-go-coordinate-free-in-monte-carlo-and-optimization","status":"publish","type":"post","link":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/","title":{"rendered":"Why Go Coordinate-Free in Monte Carlo and Optimization?"},"content":{"rendered":"<p>Traditional methods like Gibbs sampling or randomized Kaczmarz rely heavily on specific coordinate systems, which can limit their efficiency\u2014especially in ill-conditioned settings. But what happens when we step away from these constraints and adopt a coordinate-free approach?<\/p>\n<p>Our latest research examines this question, rigorously exploring the advantages and limitations of coordinate-free methods for both sampling and optimization. While Hit-and-Run has long been recognized as a coordinate-free Gibbs sampler, its quantitative benefits over coordinate-bound methods have remained largely unexplored\u2014until now.<\/p>\n<p>In collaboration with Andreas Eberle and Stefan Oberd\u00f6rster (both from the University of Bonn), we rigorously analyze and quantify the performance of coordinate-free methods like Hit-and-Run and a coordinate-free randomized Kaczmarz algorithm, identifying the settings where these methods excel.<\/p>\n<p>\u2728 Mathematical Insights<\/p>\n<ul>\n<li> <em>Ballistic and superdiffusive convergence rates<\/em>:<br \/>\nIn certain situations, coordinate-free methods achieve ballistic and superdiffusive rates, far outperforming the diffusive rates typical of coordinate-bound approaches. However, their performance is context-dependent, and they may face limitations in high-dimensional settings.<\/p>\n<li> <em>Hit-and-Run performance<\/em>:<br \/>\nWe establish tight upper and lower bounds on Wasserstein contraction rates for Hit-and-Run and derive sharp total variation mixing time upper bounds, precisely characterizing its behavior as a function of the target distribution\u2019s condition number.<\/p>\n<li> <em>Extension to randomized Kaczmarz<\/em>:<br \/>\nThese insights extend to a coordinate-free variant of the randomized Kaczmarz algorithm, a widely-used iterative method for solving linear systems. Similar convergence behavior is observed, highlighting the geometric advantages of these methods.<!--more--><\/ul>\n<\/ul>\n<p>Our findings offer a nuanced view of coordinate-free methods in sampling and optimization. While these methods demonstrate remarkable convergence rates\u2014ballistic and superdiffusive\u2014in certain scenarios, they also face limitations, particularly in high-dimensional spaces. Quantitatively understanding these trade-offs provides a clearer framework for leveraging coordinate-free approaches where they are most effective.<\/p>\n<p>\ud83d\udd17 Read the full paper here: <a href=\"https:\/\/arxiv.org\/abs\/2412.07643\">arXiv:2412.07643<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Traditional methods like Gibbs sampling or randomized Kaczmarz rely heavily on specific coordinate systems, which can limit their efficiency\u2014especially in ill-conditioned settings. But what happens when we step away from &hellip; <a href=\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/\" class=\"\">Read More<\/a><\/p>\n","protected":false},"author":2614,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[8],"tags":[],"class_list":["post-862","post","type-post","status-publish","format-standard","hentry","category-preprints"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Why Go Coordinate-Free in Monte Carlo and Optimization? - Nawaf Bou-Rabee<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Why Go Coordinate-Free in Monte Carlo and Optimization? - Nawaf Bou-Rabee\" \/>\n<meta property=\"og:description\" content=\"Traditional methods like Gibbs sampling or randomized Kaczmarz rely heavily on specific coordinate systems, which can limit their efficiency\u2014especially in ill-conditioned settings. But what happens when we step away from &hellip; Read More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/\" \/>\n<meta property=\"og:site_name\" content=\"Nawaf Bou-Rabee\" \/>\n<meta property=\"article:published_time\" content=\"2024-12-11T11:08:14+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-12-11T11:17:33+00:00\" \/>\n<meta name=\"author\" content=\"Nawaf Bou-Rabee\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nawaf Bou-Rabee\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/\",\"url\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/\",\"name\":\"Why Go Coordinate-Free in Monte Carlo and Optimization? - Nawaf Bou-Rabee\",\"isPartOf\":{\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#website\"},\"datePublished\":\"2024-12-11T11:08:14+00:00\",\"dateModified\":\"2024-12-11T11:17:33+00:00\",\"author\":{\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/f9a7363ddc14a33eef77b520728dbde0\"},\"breadcrumb\":{\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Why Go Coordinate-Free in Monte Carlo and Optimization?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#website\",\"url\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/\",\"name\":\"Nawaf Bou-Rabee\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/f9a7363ddc14a33eef77b520728dbde0\",\"name\":\"Nawaf Bou-Rabee\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f3f33e8b5e6e2430997aea3dfec1a454?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f3f33e8b5e6e2430997aea3dfec1a454?s=96&d=mm&r=g\",\"caption\":\"Nawaf Bou-Rabee\"},\"description\":\"Nawaf Bou-Rabee is a Professor of Mathematics at Rutgers University\u2013Camden, a Visiting Scholar at the Flatiron Institute, and an Adjunct Professor of Statistics and Data Science at the Wharton School, University of Pennsylvania. He works on Markov chain Monte Carlo, Bayesian computation, and the numerical analysis of stochastic differential equations, developing locally adaptive samplers such as WALNUTS, NURS, and GIST together with rigorous mixing-time and coupling theory.\",\"sameAs\":[\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/\",\"https:\/\/www.linkedin.com\/in\/nawaf-bou-rabee-559225310\"],\"url\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/author\/nb361\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Why Go Coordinate-Free in Monte Carlo and Optimization? - Nawaf Bou-Rabee","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/","og_locale":"en_US","og_type":"article","og_title":"Why Go Coordinate-Free in Monte Carlo and Optimization? - Nawaf Bou-Rabee","og_description":"Traditional methods like Gibbs sampling or randomized Kaczmarz rely heavily on specific coordinate systems, which can limit their efficiency\u2014especially in ill-conditioned settings. But what happens when we step away from &hellip; Read More","og_url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/","og_site_name":"Nawaf Bou-Rabee","article_published_time":"2024-12-11T11:08:14+00:00","article_modified_time":"2024-12-11T11:17:33+00:00","author":"Nawaf Bou-Rabee","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Nawaf Bou-Rabee","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/","url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/","name":"Why Go Coordinate-Free in Monte Carlo and Optimization? - Nawaf Bou-Rabee","isPartOf":{"@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#website"},"datePublished":"2024-12-11T11:08:14+00:00","dateModified":"2024-12-11T11:17:33+00:00","author":{"@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/f9a7363ddc14a33eef77b520728dbde0"},"breadcrumb":{"@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/why-go-coordinate-free-in-monte-carlo-and-optimization\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/"},{"@type":"ListItem","position":2,"name":"Why Go Coordinate-Free in Monte Carlo and Optimization?"}]},{"@type":"WebSite","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#website","url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/","name":"Nawaf Bou-Rabee","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/f9a7363ddc14a33eef77b520728dbde0","name":"Nawaf Bou-Rabee","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/f3f33e8b5e6e2430997aea3dfec1a454?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f3f33e8b5e6e2430997aea3dfec1a454?s=96&d=mm&r=g","caption":"Nawaf Bou-Rabee"},"description":"Nawaf Bou-Rabee is a Professor of Mathematics at Rutgers University\u2013Camden, a Visiting Scholar at the Flatiron Institute, and an Adjunct Professor of Statistics and Data Science at the Wharton School, University of Pennsylvania. He works on Markov chain Monte Carlo, Bayesian computation, and the numerical analysis of stochastic differential equations, developing locally adaptive samplers such as WALNUTS, NURS, and GIST together with rigorous mixing-time and coupling theory.","sameAs":["https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/","https:\/\/www.linkedin.com\/in\/nawaf-bou-rabee-559225310"],"url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/author\/nb361\/"}]}},"_links":{"self":[{"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/posts\/862"}],"collection":[{"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/users\/2614"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/comments?post=862"}],"version-history":[{"count":5,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/posts\/862\/revisions"}],"predecessor-version":[{"id":867,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/posts\/862\/revisions\/867"}],"wp:attachment":[{"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/media?parent=862"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/categories?post=862"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/tags?post=862"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}