2.2. Searching Google Using SOAP or SOAP with WSDL Files

Over the past few years, the SOAP standard has gathered a lot of support, in part because it is now officially documented by the World Wide Web Consortium (W3C). Standardization makes working with most SOAP services dependable, at least as far as the information you need to know and the information you can reasonably expect to get back. Because of the W3C backing, it could be said that SOAP is now the preferred architecture for most web servicesthough, despite W3C support, there's significant evidence that REST-ful services are more widely used. Nevertheless, if you're going to be working with web services, you need to be conversant with all three architectures: REST, XML-RPC, and SOAP.

SOAP's biggest drawback is its complexity, but Rails hides most of that complexity from you. Creating a SOAP client takes four simple steps:

To demonstrate, we'll build a Google search using SOAP. Like Yahoo!, Google offers a free web service API for many of their services, including their search engine. To use the Google search API (and to test the sample code), you'll need a free Google Developer's Key, which you can get directly from Google

Here's a controller that uses SOAP to find the first three results from a Google search. Update the code_controller.rb file from the previous example to contain this new googletest method:

class CodeController < ApplicationController
  def googletest
     yourkey = 'YOUR GOOGLE DEVELOPER KEY'                      # Your Google dev key
     @yourquery = 'SEARCH TEXT'                                 # Search value
     XSD::Charset.encoding = 'UTF8'                           # Set encoding for response
     googleurl = "http://api.google.com/search/beta2"
     urn = "urn:GoogleSearch"
     driver = SOAP::RPC::Driver.new(googleurl, urn)           # Create our driver
     driver.add_method('doGoogleSearch', 'key', 'q',          # Set up methods we'll call
        'start', 'maxResults', 'filter', 'restrict',
        'safeSearch', 'lr', 'ie', 'oe')
     @result = driver.doGoogleSearch(yourkey,                 # make our SOAP request
        @yourquery, 0, 3, false, '', false, '', '', '')
  end
end

Here's the code for the view. Save it as googletest.rhtml in your app/views/code/ directory:

 Query for: <%= @yourquery %><br>
 Found: <%= @result.estimatedTotalResultsCount %><br>
 Query took about <%= @result.searchTime %> seconds<br>
    <% @result.resultElements.each do |rec| %>
     <b>Title:</b> <%= rec["title"] %><br>
     <b>Summary:</b> <%= rec.snippet %><br>
     <b>Link:</b> <a href="<%= rec["URL"] %>"><%= rec["URL"] %></a>
     <br><br>
 <% end %>

And once again, we're done. We've built a complete SOAP client that you should be able to test at the URL http://localhost:3000/code/googletest.

Google Search results from our SOAP-based web service client

This client uses two additional Ruby libraries: Soap4r and XSD4R. Soap4r is a pure Ruby implementation of the SOAP 1.1 standard. For full documentation and more information, to report bugs, or to obtain the latest version of Soap4r, visit dev.ctor.org/soap4r . The XSD4R library is an XML support library that's used by Soap4r. It provides conversions between certain common data types. For complete documentation on the XSD library, see rubydoc.org/stdlib/libdoc/xsd/rdoc/index.html . Both Soap4r and XSD4R are part of the Ruby 1.8.4 distribution and should be loaded automatically; your code doesn't need to "require" them.

We start our Google example by creating an instance of a SOAP driver using the Google SOAP service URI and Namespace:

googleurl = "http://api.google.com/search/beta2"
urn = "urn:GoogleSearch"
driver = SOAP::RPC::Driver.new(googleurl, urn)

Next we define the methods we intend to invoke. The methods provided by the Google SOAP service are listed in Google's online API documentation:

driver.add_method('doGoogleSearch', 'key', 'q', 'start', 'maxResults', 'filter',
'restrict', 'safeSearch', 'lr', 'ie', 'oe')

According to the documentation, the Google web service returns all results as UTF-8 encoded values. UTF-8 can contain special characters that Ruby can't handle in a string, causing our driver to throw an XSD::ValueSpaceError error upon invocation. To avoid this problem, we manually set our encoding to UTF-8 using the XSD library.

XSD::Charset.encoding = 'UTF8'

Now our results will be properly encoded for use as native Ruby strings. The Soap4r library and the WSDL Factory library mentioned later also rely on the XSD library to handle the data-type conversions from web service results to native Ruby types.

We finish off the controller by invoking the doGoogleSearch remote method. doGoogleSearch is a method of the driver we created and configured previously:

@result = driver.doGoogleSearch(yourkey, @yourquery, 0, 3,
                                 false, '', false, '', '', '')

The view displays the results, which are returned as SOAP Mapping objects. Mapping objects allow us to access the results as either methods (for example, @result.estimatedTotalResultsCount) or as hash values (@result["estimatedTotalResultsCount"]). Because Ruby assumes that identifiers that begin with an uppercase letter are class names or constants, the method approach automatically converts the first character of each method name to lowercase. This conversion means that the method names in the Ruby code don't necessarily match the method names from the web service. Sometimes the mismatch can be awkward; for example, you could access the URL returned in the result as result.uRL, but that's very unnatural. It's therefore a good idea to use the hash approach for any methods that start with an uppercase letterfor example, rec["URL"]. In this case, using the hash is a lot more natural. Our view uses both approaches to render the results:

    Query for: <%= @yourquery %><br>
    Found: <%= @result.estimatedTotalResultsCount %><br>
    Query took about <%= @result.searchTime %> seconds<br>
    <% @result.resultElements.each do |rec| %>
     <b>Title:</b> <%= rec["title"] %><br>
     <b>Summary:</b> <%= rec.snippet %><br>
     <b>Link:</b> <a href="<%= rec["URL"] %>"><%= rec["URL"] %></a>
     <br><br>
<% end %>

Though this example is simple, it's also a bit ugly. After creating the SOAP driver, we had to call add_method to tell it all the methods we were going to call. This requirement leads to code that is inflexible, bloated, and bug-prone: will you remember to update the call to add_method if, a year from now, you add a feature that calls a new method of the API? We can solve this problem by using a WSDL file, which is an XML description of the web service's API, and the soap/wsdlfactory library. The WSDLDriverFactory downloads the WSDL file, processes it, and creates a SOAP driver that understands the service's API. However, there is one catch: Rails doesn't automatically load soap/wsdlfactory when it starts. So we've got to make sure we configure our environment to include it first. Here's how to make a SOAP client for your Rails application that uses a WSDL file:

Let's refactor our Google example to use a WSDL file, instead of manually defining the methods we want to call. Start by updating environment.rb:

require 'soap/wsdlDriver'

Then restart your Webrick server, so that it picks up this change to the environment. Now you're ready for the WSDL-enabled version of the controller:

class CodeController < ApplicationController
  def googletest
     yourkey = 'YOUR GOOGLE DEVELOPER KEY'                # Your Google dev key
     @yourquery = 'SEARCH TEXT'                           # Search value
     XSD::Charset.encoding = 'UTF8'                       # Set encoding
     wsdlfile = "http://api.google.com/GoogleSearch.wsdl"     # WSDL location
     driver = SOAP::WSDLDriverFactory.new(wsdlfile).create_rpc_driver     # Create driver
                                                                          # and set up
                                                                          # methods
     @result = driver.doGoogleSearch(yourkey, @yourquery, # make our SOAP request
                 0, 3, false, '', false, '', '', '')
     end
end

The only real change is that we create the driver using a WSDL file that's provided by the Google web service, rather by specifying an endpoint URI and namespace:

driver = SOAP::WSDLDriverFactory.new(wsdlfile).create_rpc_driver

We've removed the call to driver.add_method, since we no longer need to specify which methods we intend to invoke; that information now comes from the WSDL file. Of course, this doesn't save us from reading the web service's documentation; we still need to know the required parameters and the returned values of any methods we call.

Nothing we've done requires any changes to the view, so you can test this new application at the URL http://localhost:3000/code/googletest. Complete documentation of the search features and the WSDL file used in our example can be found at google.com/apis/.

We have just one more type of web service client to cover, so let's shift our focus to XML-RPC and build a client for the very popular Flickr service.