{"id":888,"date":"2025-06-24T17:28:48","date_gmt":"2025-06-24T17:28:48","guid":{"rendered":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/?p=888"},"modified":"2025-06-26T12:04:38","modified_gmt":"2025-06-26T12:04:38","slug":"walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler","status":"publish","type":"post","link":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/","title":{"rendered":"WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler"},"content":{"rendered":"<p>\nIn many Bayesian inference problems, the geometry of the posterior distribution can vary dramatically in scale. A classic example is Neal\u2019s funnel, where the state-of-the-art algorithm, the No-U-Turn Sampler (NUTS), struggles due to its use of a fixed leapfrog step size.\n<\/p>\n<p>\nWe&#8217;ve just released a new solution: <strong>WALNUTS<\/strong> (<strong>W<\/strong>ithin-orbit <strong>A<\/strong>daptive <strong>L<\/strong>eapfrog <strong>N<\/strong>o-<strong>U<\/strong>&#8211;<strong>T<\/strong>urn <strong>S<\/strong>ampler), and yes, the acronym did come first. WALNUTS builds on over a decade of progress in locally adaptive Hamiltonian Monte Carlo. It adapts the step size <em>within<\/em> each orbit\u2014not between orbits, but inside them (see image below). Crucially, it preserves the key features of NUTS: path-length adaptivity and biased progressive sampling.\n<\/p>\n<p>\nTo do this, WALNUTS introduces a macro\/micro timescale:\n<\/p>\n<ul>\n<li><strong>Macro steps<\/strong> define the coarse skeleton of the orbit, i.e., the orbit is built by advancing from one macro point to the next.<\/li>\n<li><strong>Micro steps<\/strong> adapt dynamically between macro points, refining the integration resolution to control local energy error.<\/li>\n<\/ul>\n<div style=\"max-width: 650px;margin: 30px auto;padding: 16px;border: 1px solid #ddd;border-radius: 8px;background-color: #f9f9f9;text-align: center\">\n  <img decoding=\"async\" src=\"http:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png\" alt=\"WALNUTS transition step in Neal\u2019s funnel\" style=\"width: 100%;height: auto;border-radius: 4px\" \/><\/p>\n<div style=\"margin-top: 10px;font-size: 0.95em;color: #444\">\n    <em>Figure:<\/em> WALNUTS transition step in Neal\u2019s funnel.<br \/>\n    White dots mark macro step positions. <br \/> Colored bands represent variable-resolution leapfrog integration. <br \/> The star marks the final state, selected via biased progressive sampling using the weights shown in the inset.\n  <\/div>\n<\/div>\n<p>\nThe end result is robust, efficient sampling\u2014even deep in the neck of Neal\u2019s funnel.  See, e.g., Figure 14 in the preprint below where we drop WALNUTS into one of the most ill-conditioned parts of the target, and yet even from this worst-case initialization, WALNUTS stabilizes, adapts, and escapes \u2013 without manual tuning.  That&#8217;s the strength of within-orbit adaptivity.\n<\/p>\n<p>\n\ud83d\udd17 <a href=\"https:\/\/arxiv.org\/abs\/2506.18746\">Paper<\/a> | \ud83d\udd17 <a href=\"https:\/\/github.com\/bob-carpenter\/walnuts\">Code<\/a>\n<\/p>\n<p>\n\ud83d\ude4c A fun and rewarding collaboration with Bob Carpenter (Flatiron), Tore Selland Kleppe (Norway), and Sifan Liu (Flatiron) \u2014 supported by the Flatiron Institute and Rutgers University.\n<\/p>\n<p>\nBig thanks to Brian Ward and Steve Bronder for their contributions to a fast and efficient C++ implementation of WALNUTS, and for their help integrating it into the Stan ecosystem.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In many Bayesian inference problems, the geometry of the posterior distribution can vary dramatically in scale. A classic example is Neal\u2019s funnel, where the state-of-the-art algorithm, the No-U-Turn Sampler (NUTS), &hellip; <a href=\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/\" class=\"\">Read More<\/a><\/p>\n","protected":false},"author":2614,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[8],"tags":[],"class_list":["post-888","post","type-post","status-publish","format-standard","hentry","category-preprints"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler - Nawaf Bou-Rabee<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler - Nawaf Bou-Rabee\" \/>\n<meta property=\"og:description\" content=\"In many Bayesian inference problems, the geometry of the posterior distribution can vary dramatically in scale. A classic example is Neal\u2019s funnel, where the state-of-the-art algorithm, the No-U-Turn Sampler (NUTS), &hellip; Read More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/\" \/>\n<meta property=\"og:site_name\" content=\"Nawaf Bou-Rabee\" \/>\n<meta property=\"article:published_time\" content=\"2025-06-24T17:28:48+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-26T12:04:38+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png\" \/>\n<meta name=\"author\" content=\"Nawaf Bou-Rabee\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nawaf Bou-Rabee\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/\",\"url\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/\",\"name\":\"WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler - Nawaf Bou-Rabee\",\"isPartOf\":{\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png\",\"datePublished\":\"2025-06-24T17:28:48+00:00\",\"dateModified\":\"2025-06-26T12:04:38+00:00\",\"author\":{\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/f9a7363ddc14a33eef77b520728dbde0\"},\"breadcrumb\":{\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#primaryimage\",\"url\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png\",\"contentUrl\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png\",\"width\":741,\"height\":683},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#website\",\"url\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/\",\"name\":\"Nawaf Bou-Rabee\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/f9a7363ddc14a33eef77b520728dbde0\",\"name\":\"Nawaf Bou-Rabee\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f3f33e8b5e6e2430997aea3dfec1a454?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f3f33e8b5e6e2430997aea3dfec1a454?s=96&d=mm&r=g\",\"caption\":\"Nawaf Bou-Rabee\"},\"description\":\"Math Professor at Rutgers\",\"sameAs\":[\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/\"],\"url\":\"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/author\/nb361\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler - Nawaf Bou-Rabee","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/","og_locale":"en_US","og_type":"article","og_title":"WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler - Nawaf Bou-Rabee","og_description":"In many Bayesian inference problems, the geometry of the posterior distribution can vary dramatically in scale. A classic example is Neal\u2019s funnel, where the state-of-the-art algorithm, the No-U-Turn Sampler (NUTS), &hellip; Read More","og_url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/","og_site_name":"Nawaf Bou-Rabee","article_published_time":"2025-06-24T17:28:48+00:00","article_modified_time":"2025-06-26T12:04:38+00:00","og_image":[{"url":"http:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png"}],"author":"Nawaf Bou-Rabee","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Nawaf Bou-Rabee","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/","url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/","name":"WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler - Nawaf Bou-Rabee","isPartOf":{"@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#website"},"primaryImageOfPage":{"@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#primaryimage"},"image":{"@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#primaryimage"},"thumbnailUrl":"http:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png","datePublished":"2025-06-24T17:28:48+00:00","dateModified":"2025-06-26T12:04:38+00:00","author":{"@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/f9a7363ddc14a33eef77b520728dbde0"},"breadcrumb":{"@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#primaryimage","url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png","contentUrl":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-content\/uploads\/sites\/908\/2025\/06\/funnel_walnuts_amin_0p97_oistr_1.png","width":741,"height":683},{"@type":"BreadcrumbList","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/walnuts-within-orbit-adaptive-leapfrog-no-u-turn-sampler\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/"},{"@type":"ListItem","position":2,"name":"WALNUTS = Within-orbit Adaptive Leapfrog No-U-Turn Sampler"}]},{"@type":"WebSite","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#website","url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/","name":"Nawaf Bou-Rabee","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/f9a7363ddc14a33eef77b520728dbde0","name":"Nawaf Bou-Rabee","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/f3f33e8b5e6e2430997aea3dfec1a454?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f3f33e8b5e6e2430997aea3dfec1a454?s=96&d=mm&r=g","caption":"Nawaf Bou-Rabee"},"description":"Math Professor at Rutgers","sameAs":["https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/"],"url":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/author\/nb361\/"}]}},"_links":{"self":[{"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/posts\/888"}],"collection":[{"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/users\/2614"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/comments?post=888"}],"version-history":[{"count":24,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/posts\/888\/revisions"}],"predecessor-version":[{"id":913,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/posts\/888\/revisions\/913"}],"wp:attachment":[{"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/media?parent=888"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/categories?post=888"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.rutgers.edu\/nawaf-bou-rabee\/wp-json\/wp\/v2\/tags?post=888"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}