Multi Request HTTP or HTTPS With Typhoeus Gem

Multi Request HTTP or HTTPS With Typhoeus Gem

There are many kinds of complexity that you have to deal with developing software and different kinds of applications will have very different sets of problems you need to solve. With this article, I want to show you about one Technich of Gem that is called Typhoeus . It can say that it is the best one that i never found before. If you think you have any problem as below:

• Make the request with many URL in the same time
• Make the download website with multi thread like http://en.savefrom.net/
• Make requests http or https in parallel while cleanly encapsulating handling logic.

Here we are, With Typhoeus is specifically designed for building RESTful service oriented architectures in Ruby that need to be fast enough to process calls to multiple services within the client’s HTTP request/response life cycle.

Some of the awesome features are parallel request execution, memoization of request responses (so you don’t make the same request multiple times in a single group), built in support for caching responses to memcached (or whatever), and mocking capability baked in. It uses libcurl and libcurl-multi to work this speedy magic. I wrote the bindings myself so it’s yet another Ruby libcurl library, but with some extra awesomeness added in. FFI is used to interface with the library so it works with any Ruby implementation.

Installation

It is not difficult to install it with your project just run it with this code:

 gem install typhoeus

But it's sometime if you have some problem with installation with this gem, you can troubleshooting. Do this to fix: (only on Ubutun or Debian)

sudo apt-get install libcurl4-gnutls-dev

By the way, Tyhoeus gem also support too on Windows, It runs perfectly on Windows, But you need to install Libcurl on your Window first before you install Typhoeus. With this link http://curl.haxx.se/download.html#Win32 you can choose the version of Libcurl for your Windows.

Usage

I will show you guy about simple example about Typhoeus, However the primary interface for Typhoeus is comprised of three classes: Request, Response, and Hydra. Request represents an HTTP request object, response represents an HTTP response, and Hydra manages making parallel HTTP connections.

Ok, let's start create with new project in ruby on rails:

 $ rails new download_project

 # ...

 $ cd download_project

Now add typhoeus to your Gemfile

source "https://rubygems.org"

gem "rails", "4.2.4"
gem "mysql2", "~> 0.3.20"
gem "sass-rails", "~> 5.0"
gem "uglifier", ">= 1.3.0"
gem "coffee-rails", "~> 4.1.0"
gem "jquery-rails"
gem "turbolinks"
gem "jbuilder", "~> 2.0"
gem "sdoc", "~> 0.4.0", group: :doc
gem "bootstrap-sass", "~> 3.3.6"
gem "typhoeus"

group :development, :test do
  gem "byebug"
end

group :development do
  gem "pry"
  gem "web-console", "~> 2.0"
  gem "spring"
end

After that you need to run install your gem

$ bundle install

you need to create controller and view for this example project by ruby running this code below:

$ rails generate controller downloads index

Go into our project and create new folder as [your_project_name]/app/services and after run this code to create new file that name is download.rb

touch /app/services/download.rb

after you run that code above you can see as this in your project:

Open your file download.rb and edit to add code below:

class Download
  def get_video url
    downloaded_file = File.open "example.mp4", "wb"
    hydra = Typhoeus::Hydra.new max_concurrency: 100
    request = Typhoeus::Request.new url

    request.on_body do |chunk|
      downloaded_file.write chunk
    end
    request.on_complete do |response|
      downloaded_file.close
    end
    hydra.queue request

    hydra.run
  end
end

Note: Typhoeus can stream responses. When you're expecting a large response, set the on_body callback on a request. Typhoeus will yield to the callback with chunks of the response, as they're read. When you set an on_body callback, Typhoeus will not store the complete response.

After that let open our file ** app/views/downloads/index.html.erb** and edit to add as this:

<div class="container">
  <label for="basic-url">Your vanity URL</label>
  <%= form_tag downloads_path, method: :post, class: "form-inline" do %>
    <div class="input-group form-group">
      <span class="input-group-addon" id="basic-addon3">youtube</span>
      <%= text_field_tag "url", nil, class: "form-control", id: "basic-url",
        "aria-describedby": "basic-addon3" %>
    </div>
    <%= submit_tag "download", class: "btn btn-success" %>
  <% end %>
</div>

It is not complete yet to run it, you need to edit more with your controller file: app/controllers/downloads_controller.rb :

class DownloadsController < ApplicationController
  def index
  end

  def create
    url = params[:url]
    download = Download.new
    download.get_video url
    redirect_to downloads_path
  end
end

This is finaly , just run your application with:

$ rails server

you will see this (don't forget to configure your file config/routes.rb)

example copy link url: http://vjs.zencdn.net/v/oceans.mp4 and click download you will get file mp4 from that url. After that you can check it in your project you will get it as this image.

Conclusion

For the end, with this article it is just simple example with typhoeus gem but it is very important to how to the basic is better to learn difficult thing to understand. And with this to you can to make the download website to get downloading video moviehdkh, youtube, facebook and other websites. However, it all depends on internet connection, server latency and CPU speed. The difference is vague though since it’s hard to rely on test results because of many outside factors involved. Personally I find typhoeus being the best solution for downloading multiple files because it’s based on robust libcurl and it’s pretty easy to get started with. But if you don’t like installing custom gem.

Documents

• https://github.com/typhoeus/typhoeus
• Service-Oriented Design With Ruby On rails