# Technical Documentation: rubycdp/ferrum > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/rubycdp/ferrum) ### Overview (`README.md`) # Ferrum - high-level API to control Chrome in Ruby ## [Documentation](https://docs.rubycdp.com/docs/ferrum/introduction)

It is Ruby clean and high-level API to Chrome. Runs headless by default, but you can configure it to run in a headful mode. All you need is Ruby and [Chrome](https://www.google.com/chrome/) or [Chromium](https://www.chromium.org/). Ferrum connects to the browser by [CDP protocol](https://chromedevtools.github.io/devtools-protocol/) and there's _no_ Selenium/WebDriver/ChromeDriver dependency. The emphasis was made on a raw CDP protocol because Chrome allows you to do so many things that are barely supported by WebDriver because it should have consistent design with other browsers.
## Installation There's no official Chrome or Chromium package for Linux don't install it this way because it's either outdated or unofficial, both are bad. Download it from official source for [Chrome](https://www.google.com/chrome/) or [Chromium](https://www.chromium.org/getting-involved/download-chromium). Chrome binary should be in the `PATH` or `BROWSER_PATH` and you can pass it as an option to browser instance see `:browser_path` in [Customization](/docs/ferrum/customization). Add this to your `Gemfile` and run `bundle install`. ``` ruby gem "ferrum" ``` ## Docker To skip installing Chrome, run it in a container with our images, `ghcr.io/rubycdp/chrome` or `ghcr.io/rubycdp/chromium`, and connect with `Ferrum::Browser.new(url: "http://localhost:3000")`. See [Docker](/docs/ferrum/docker) for that and for running Chrome in your own container. ## Quick Start Navigate to a website and save a screenshot: ```ruby browser = Ferrum::Browser.new browser.go_to("https://google.com") browser.screenshot(path: "google.png") browser.quit ``` When you work with browser instance Ferrum creates and maintains a default page for you, in fact all the methods above are sent to the `page` instance that is created in the `default_context` of the `browser` instance. You can interact with a page created manually and this is preferred: ```ruby browser = Ferrum::Browser.new page = browser.create_page page.go_to("https://google.com") input = page.at_xpath("//input[@name='q']") input.focus.type("Ruby headless driver for Chrome", :Enter) page.at_css("a > h3").text # => "rubycdp/ferrum: Ruby Chrome/Chromium driver - GitHub" browser.quit ``` Evaluate some JavaScript and get full width/height: ```ruby browser = Ferrum::Browser.new page = browser.create_page page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara") width, height = page.evaluate <<~JS [document.documentElement.offsetWidth, document.documentElement.offsetHeight] JS # => [1024, 1931] browser.quit ``` Do any mouse movements you like: ```ruby # Trace a 100x100 square browser = Ferrum::Browser.new page = browser.create_page page.go_to("https://google.com") page.mouse .move(x: 0, y: 0) .down .move(x: 0, y: 100) .move(x: 100, y: 100) .move(x: 100, y: 0) .move(x: 0, y: 0) .up browser.quit ``` ## Clean Up Closes browser tabs opened by the `Browser` instance. ```ruby # connect to a long-running Chrome process browser = Ferrum::Browser.new(url: "http://localhost:9222") browser.go_to("https://github.com/") # clean up, lest the tab stays there hanging forever browser.reset browser.quit ``` `#quit` blocks by default until the browser process is confirmed dead and its user data directory removed. Killing a stubborn process (one that ignores `TERM` and needs `KILL`) can take a couple of seconds, which matters if you're quitting many browsers in a hot path. Pass `wait: false` to return immediately and run that cleanup on a background thread instead: ```ruby browser = Ferrum::Browser.new thread = browser.quit(wait: false) # ... do other work while the browser is killed and its directory removed in the background ... thread.join # only needed if you must wait for cleanup to finish, e.g. before reusing a fixed port ``` ## Thread safety Ferrum is fully thread-safe. You can create one browser or a few as you wish and start playing around using threads. Example below shows how to create a few pages which share the same context. Context is similar to an incognito profile but you can have more than one, think of it like it's independent browser session: ```ruby browser = Ferrum::Browser.new context = browser.contexts.create t1 = Thread.new(context) do |c| page = c.create_page page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara") page.screenshot(path: "t1.png") end t2 = Thread.new(context) do |c| page = c.create_page page.go_to("https://www.google.com/search?q=Ruby+static+typing") page.screenshot(path: "t2.png") end t1.join t2.join context.dispose browser.quit ``` or you can create two independent contexts: ```ruby browser = Ferrum::Browser.new t1 = Thread.new(browser) do |b| context = b.contexts.create page = context.create_page page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara") page.screenshot(path: "t1.png") context.dispose end t2 = Thread.new(browser) do |b| context = b.contexts.create page = context.create_page page.go_to("https://www.google.com/search?q=Ruby+static+typing") page.screenshot(path: "t2.png") context.dispose end t1.join t2.join browser.quit ``` ## Development After checking out the repo, run `bundle install` to install dependencies. Then, run `bundle exec rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment. To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org). ## Contributing Bug reports and pull requests are welcome on [GitHub](https://github.com/rubycdp/ferrum). --- ### 10 Mouse Keyboard (`docs/10-mouse-keyboard.md`) --- sidebar_position: 10 --- # Mouse & Keyboard ## Mouse `page.mouse` #### scroll_to(x, y) Scroll page to a given x, y * x `Integer` the pixel along the horizontal axis of the document that you want displayed in the upper left * y `Integer` the pixel along the vertical axis of the document that you want displayed in the upper left ```ruby page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara") page.mouse.scroll_to(0, 400) ``` #### click(\*\*options) : `Mouse` Click given coordinates, fires mouse move, down and up events. * options `Hash` * :x `Integer` * :y `Integer` * :delay `Float` defaults to 0. Delay between mouse down and mouse up events * :button `Symbol` :left | :right, defaults to :left * :count `Integer` defaults to 1 * :modifiers `Integer` bitfield for key modifiers. See `keyboard.modifiers`. If omitted, uses the keys held by `keyboard.hold`, or none outside of it #### down(\*\*options) : `Mouse` Mouse down for given coordinates. * options `Hash` * :button `Symbol` :left | :right, defaults to :left * :count `Integer` defaults to 1 * :modifiers `Integer` bitfield for key modifiers. See `keyboard.modifiers`. If omitted, uses the keys held by `keyboard.hold`, or none outside of it #### up(\*\*options) : `Mouse` Mouse up for given coordinates. * options `Hash` * :button `Symbol` :left | :right, defaults to :left * :count `Integer` defaults to 1 * :modifiers `Integer` bitfield for key modifiers. See `keyboard.modifiers`. If omitted, uses the keys held by `keyboard.hold`, or none outside of it #### move(x:, y:, steps: 1, modifiers: nil) : `Mouse` Mouse move to given x and y. * options `Hash` * :x `Integer` * :y `Integer` * :steps `Integer` defaults to 1. Sends intermediate mousemove events. * :modifiers `Integer` bitfield for key modifiers. See `keyboard.modifiers`. If omitted, uses the keys held by `keyboard.hold`, or none outside of it ## Keyboard `page.keyboard` #### down(key) : `Keyboard` Dispatches a keydown event. * key `String` | `Symbol` Name of key such as "a", :enter, :backspace. Raises `ArgumentError` for a modifier, use `hold` instead #### up(key) : `Keyboard` Dispatches a keyup event. * key `String` | `Symbol` Name of key such as "b", :enter, :backspace. Raises `ArgumentError` for a modifier, use `hold` instead #### type(\*keys) : `Keyboard` Sends a keydown, keypress/input, and keyup event for each character in the text. * text `String` | `Array | Array` A text to type into a focused element, `[:Shift, "s"], "tring"` #### hold(\*keys) { } : `Object` Holds modifier keys down while the block runs. Every keyboard and mouse event inside the block carries them, and they are released when the block returns or raises. Returns the block's value. * keys `Symbol` :alt | :ctrl | :control | :meta | :command | :shift ```ruby page.keyboard.hold(:shift) do page.mouse.move(x: 10, y: 10).down.move(x: 200, y: 200, steps: 10).up end ``` #### held_modifiers : `Integer` Modifier keys currently pressed by `hold`, in the same format as `modifiers` returns. It's 0 outside a `hold` block. ```ruby page.keyboard.hold(:shift) { page.keyboard.held_modifiers } # => 8 ``` #### modifiers(keys) : `Integer` Returns bitfield for a given keys * keys `Array` :alt | :ctrl | :command | :shift ## Examples ```ruby # Type text into an input page.go_to("https://google.com") input = page.at_css("input[name='q']") input.focus page.keyboard.type("Hello World") # Press Enter page.keyboard.type(:Enter) # Use keyboard shortcuts page.keyboard.hold(:shift) { page.keyboard.type("hello") } ``` --- ### 11 Cookies (`docs/11-cookies.md`) --- sidebar_position: 11 --- # Cookies `page.cookies` #### all : `Hash` Returns cookies hash ```ruby page.cookies.all # => {"NID"=>#"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>} ``` #### \[\](value) : `Cookie` Returns cookie * value `String` ```ruby page.cookies["NID"] # => "NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}> ``` #### set(value) : `Boolean` Sets a cookie * value `Hash` * :name `String` * :value `String` * :domain `String` * :expires `Integer` * :samesite `String` * :httponly `Boolean` ```ruby page.cookies.set(name: "stealth", value: "omg", domain: "google.com") # => true ``` * value `Cookie` ```ruby nid_cookie = page.cookies["NID"] # => page.cookies.set(nid_cookie) # => true ``` #### remove(\*\*options) : `Boolean` Removes given cookie * options `Hash` * :name `String` * :domain `String` * :url `String` ```ruby page.cookies.remove(name: "stealth", domain: "google.com") # => true ``` #### clear : `Boolean` Removes all cookies for current page ```ruby page.cookies.clear # => true ``` #### store(path) : `Boolean` Stores all cookies of current page in a file. ```ruby # Cookies are saved into cookies.yml page.cookies.store # => 15657 ``` #### load(path) : `Boolean` Loads all cookies from the file and sets them for current page. ```ruby # Cookies are loaded from cookies.yml page.cookies.load # => true ``` --- ### 12 Headers (`docs/12-headers.md`) --- sidebar_position: 12 --- # Headers `page.headers` #### get : `Hash` Get all headers #### set(headers) : `Boolean` Set given headers. Eventually clear all headers and set given ones. * headers `Hash` key-value pairs for example `"User-Agent" => "Browser"` #### add(headers) : `Boolean` Adds given headers to already set ones. * headers `Hash` key-value pairs for example `"Referer" => "http://example.com"` #### clear : `Boolean` Clear all headers. --- ### 13 Javascript (`docs/13-javascript.md`) --- sidebar_position: 13 --- # JavaScript #### evaluate(expression, \*args, timeout: nil, \*\*named) Evaluate JavaScript and return the result, converted to Ruby. * expression `String` JavaScript returning the value of its last statement, or a function/arrow declaration, which is used as-is * args positional arguments, a declaration's parameters or `arguments[0]` and up. Can't be mixed with named ones * timeout `Numeric` seconds to wait for the script and any promise it returns, defaults to the page timeout. Raises `Ferrum::ScriptTimeoutError` when exceeded, which inherits from `Ferrum::TimeoutError` * named named arguments, which become the parameters of a script that isn't a declaration. A `Ferrum::Node` arrives as the live element, but can't be put inside an Array or Hash. `timeout` can't be a parameter name ```ruby page.evaluate("[window.scrollX, window.scrollY]") # => [0, 0] page.evaluate("a + b", a: 1, b: 2) # => 3 page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href") page.evaluate("await fetch(url).then(r => r.status)", url: "/") # => 200 page.evaluate(<<~JS, 3) # => 9 function(c) { const sum = 1 + 2; return sum * c; } JS ``` The result is converted to Ruby: objects become `Hash`, arrays and node lists `Array`, DOM nodes `Ferrum::Node`, dates ISO 8601 strings, `NaN` and `Infinity` `Float`, `BigInt` `Integer`, `null` and `undefined` `nil`. Values without a Ruby counterpart, such as functions, `Map`, `Set` or `window`, become `{}`. A reference back to an enclosing object, e.g. `a.self = a`, becomes `Ferrum::CyclicObject`. #### evaluate_handle(expression, \*args, timeout: nil, \*\*named) : `Ferrum::RemoteObject` Same as `evaluate`, but keeps the value in the browser and returns a reference you can pass back in as an argument. Call `#value` to convert it to Ruby, and `#release` to free it. ```ruby list = page.evaluate_handle("document.querySelectorAll('li')") page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list) ``` #### execute(expression, \*args, timeout: nil, \*\*named) : `Boolean` Run JavaScript for its side effects. The script is used as a function body, so statements need no wrapping. ```ruby page.execute("window.scrollBy(0, 100)") # => true ``` `Ferrum::Node` has the same three methods, with `this` bound to the element. Arrow functions don't get `this`, so use a `function` declaration when you need it. ```ruby page.at_css("input").evaluate("this.value") page.at_css("input").execute("this.value = text", text: "hello") ``` #### Upgrading to 0.21 Scripts written for the old API run exactly as before, and Ferrum warns, pointing at your code, about any whose result will change. Calls that only the new API can make, e.g. with named arguments, already run the new way. * 0.20 runs old-style scripts the old way and warns. Set `FERRUM_JS_API=new` to run everything the new way. * 0.21 removes the old way, along with `evaluate_async`, `evaluate_func` and `evaluate_on`. There's no switch back. Warnings only cover the scripts and results your suite actually runs into, so before upgrading to 0.21, run the suite once with `FERRUM_JS_API=new`. Set `FERRUM_DEPRECATION_WARNINGS=raise` to turn the warnings into errors while migrating, or `0` to silence them. A library built on Ferrum should pass named arguments or a function declaration, which run the new way in both 0.20 and 0.21. | Script | < 0.21 | >= 0.21 | Notes | |---|---|---|---| | `evaluate("arguments[0].foo", foo: 1)` | `1` | `nil` | keywords are named parameters now, write `evaluate("foo", foo: 1)` | | `evaluate("[arguments[0], arguments[1]]", 1, foo: 2)` | `[1, { "foo" => 2 }]` | `ArgumentError` | positional and keyword arguments can't be mixed, pass `{ foo: 2 }` in braces | | `evaluate("a(); b")` | `a()`'s value, `b` never runs | `b`'s value | every statement runs, the last one's value is returned | | `evaluate("\n window.ran = true\n")` | `nil`, nothing runs | `true` | the line break no longer ends the `return`, check it's safe to run | | `evaluate("Promise.resolve(1)")` | `{}` | `1` | promises are awaited, `void` keeps one from being awaited | | `node.evaluate("this.parentNode")` | `{}` | `Ferrum::Node` | | | `node.evaluate("new Date(0)")` | `{}` | `"1970-01-01T00:00:00.000Z"` | | | `evaluate_async("arguments[0](1)", 5)` | `1` | `NoMethodError` | `evaluate("await …", timeout: 5)` | | `evaluate_func("function(a) { return a }", 1)` | `1` | `NoMethodError` | `evaluate("function(a) { return a }", 1)`, with `on:` use `node.evaluate` | | `evaluate_on(node: node, expression: "this.id")` | `"main"` | `NoMethodError` | `node.evaluate("this.id")` | | `evaluate_on(node: link, expression: "this.click()", wait: 0.1)` | waits for the new page | `NoMethodError` | `page.wait_for_navigation!(0.1) { link.evaluate("this.click()") }` | #### evaluate_on_new_document(source, \*\*options) : `String` Runs JavaScript in every new document of the page and its in-process frames, before the document's own scripts. Returns the script identifier. For every page and iframe of the browser, use `add_extension`. * source `String` should be valid JavaScript * options `Hash` * :world_name `String` - run in an isolated world with this name instead of the page's main world * :run_immediately `Boolean` - run in the current document as well, `false` by default ```ruby id = page.evaluate_on_new_document <<~JS Object.defineProperty(navigator, "languages", { get: function() { return ["tlh"]; } }); JS ``` #### isolated_world(name) : `IsolatedWorld` Returns the frame's isolated world with this name, its own JavaScript globals over the frame's DOM, so the page's scripts can't see or override what runs there. It has `evaluate`, `evaluate_handle` and `execute`, takes and returns nodes, and comes back empty in every new document. On a page it's the main frame's world, use `frame.isolated_world(name)` for an iframe. It's the same world `evaluate_on_new_document` runs in with `world_name:`. * name `String` ```ruby world = page.isolated_world("ferrum") world.execute("window.secret = 'top'") world.evaluate("window.secret") # => "top" page.evaluate("window.secret") # => nil world.evaluate("document.title") # => same as page.evaluate("document.title") ``` #### remove_script_on_new_document(identifier) : `Boolean` Stops running a script added with `evaluate_on_new_document` in new documents. ```ruby page.remove_script_on_new_document(id) # => true ``` #### add_extension(extension) Adds an extension, see the `:extensions` option. It runs in every page and iframe, popups and out-of-process iframes included, attached after the call, before their own scripts. Pages already open keep the extensions they had. * extension `String | Hash` a path to a JS file or `{ source: "..." }` ```ruby browser.add_extension("/path/to/stub.js") browser.add_extension(source: "window.secret = 'top'") ``` #### add_script_tag(\*\*options) : `Boolean` * options `Hash` * :url `String` * :path `String` * :content `String` * :type `String` - `text/javascript` by default ```ruby page.add_script_tag(url: "http://example.com/stylesheet.css") # => true ``` #### add_style_tag(\*\*options) : `Boolean` * options `Hash` * :url `String` * :path `String` * :content `String` ```ruby page.add_style_tag(content: "h1 { font-size: 40px; }") # => true ``` #### bypass_csp(\*\*options) : `Boolean` * options `Hash` * :enabled `Boolean`, `true` by default ```ruby page.bypass_csp # => true page.go_to("https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/promises.in.md") page.refresh page.add_script_tag(content: "window.__injected = 42") page.evaluate("window.__injected") # => 42 ``` #### disable_javascript Disables Javascripts from the loaded HTML source. You can still evaluate JavaScript with `evaluate` or `execute`. Returns nothing. ```ruby page.disable_javascript ``` ## Console #### console : `Console` What the DevTools console would show for the page: `console.*` calls, browser log entries (failed resource loads, CSP violations, deprecations, ...) and uncaught exceptions. It keeps the last 1000 messages, oldest first, and is `Enumerable`, so filter it like any collection. ```ruby page.go_to("https://example.com") page.console.select(&:error?) # => [#] page.console.select { |message| message.match?(/deprecated/) } page.console.last.text # => "Hello world" page.console.clear ``` `Ferrum::Console::Message` has: * `type` `Symbol` - `:log`, `:info`, `:warning`, `:error`, `:debug`, `:table`, ... for `console.*` calls, the log entry's level for browser log entries, `:error` for uncaught exceptions * `source` `Symbol` - `:console_api`, `:exception`, or the log entry's source such as `:network` or `:security` * `text` `String` - the arguments joined by spaces, objects and arrays as JSON * `args` `Array` - the `console.*` arguments as Ruby values, objects and arrays built from Chrome's shallow preview * `location` `Location` and `stack_trace` `Array` - `url`, `line`, `column`, `function_name` * `timestamp` `Time` * `severity` `Symbol` - `:debug`, `:info`, `:warn` or `:error`, the matching `Logger` method * `error?`, `warning?`, `match?(pattern)` #### on(:console) Calls the block for every new message. ```ruby logger = Logger.new($stdout) page.on(:console) { |message| logger.public_send(message.severity, message.to_s) } ``` Fail a test on browser errors: ```ruby after { expect(browser.console.select(&:error?)).to be_empty } ``` --- ### 14 Emulation (`docs/14-emulation.md`) --- sidebar_position: 14 --- # Emulation #### set_viewport Overrides device screen dimensions and emulates viewport. When `:mobile` is `true`, this also enables touch emulation (`navigator.maxTouchPoints`, `ontouchstart`) alongside the mobile viewport metrics. * options `Hash` * :width `Integer`, viewport width. `0` by default * :height `Integer`, viewport height. `0` by default * :scale_factor `Float`, device scale factor. `0` by default * :mobile `Boolean`, whether to emulate mobile device and enable touch. `false` by default ```ruby page.set_viewport(width: 1000, height: 600, scale_factor: 3) ``` `:width`, `:height`, and `:scale_factor` all default to `0`, which per the [DevTools protocol](https://chromedevtools.github.io/devtools-protocol/tot/Emulation/#method-setDeviceMetricsOverride) disables overriding that particular value. This means `mobile:` can be enabled on its own, at the page's current size, without specifying a device size preset: ```ruby page.set_viewport(mobile: true) ``` --- ### 15 Frames (`docs/15-frames.md`) --- sidebar_position: 15 --- # Frames Frames allow you to work with iframes and nested documents within a page. Each frame has its own execution context and can be interacted with independently. #### frames : `Array[Frame] | []` Returns all the frames current page have. ```ruby page.go_to("https://www.w3schools.com/tags/tag_frame.asp") page.frames # => # [ # #, # #, # #, # ... # ] ``` #### main_frame : `Frame` Returns page's main frame, the top of the tree and the parent of all frames. #### frame_by(\*\*options) : `Frame | nil` Find frame by given options. * options `Hash` * :id `String` - Unique frame's id that browser provides * :name `String` - Frame's name if there's one ```ruby page.frame_by(id: "C6D104CE454A025FBCF22B98DE612B12") ``` ## Frame ## Frame #### id : `String` Frame's unique id. #### parent_id : `String | nil` Parent frame id if this one is nested in another one. #### parent : `Frame | nil` Parent frame if this one is nested in another one. #### frame_element : `Node | nil` Returns the element in which the window is embedded. It's `nil` for the main frame and for a frame from a different origin than its parent, since the same-origin policy hides `window.frameElement` from a cross-origin frame. #### execution_id : `Integer` Execution context id which is used by JS, each frame has its own context in which JS evaluates. #### name : `String | nil` If frame was given a name it should be here. #### state : `Symbol | nil` One of the states frame's in: * `:started_loading` * `:navigated` * `:stopped_loading` #### loader_id : `String | nil` Id of the loader associated with the frame's current navigation, set once the frame navigates. #### lifecycle_events : `Array[Hash]` List of the frame's lifecycle events as reported by the browser, each a `{"name" => String, "timestamp" => Float}` hash, e.g. `init`, `networkIdle`, `firstPaint`. #### idle? : `Boolean` Whether the frame has finished loading (`:stopped_loading` state). Frames in `:canceled` state (execution context torn down mid-navigation) are not considered idle. ```ruby page.go_to("https://example.com") page.main_frame.idle? # => true ``` #### url : `String` Returns current frame's location href. ```ruby page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe") frame = page.frames[1] frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.html ``` #### title Returns current frame's title. ```ruby page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe") frame = page.frames[1] frame.title # => HTML Demo: