Quick Start Guide with SOAP Part Two

Table of Contents

Quick Start with SOAP Part I

More Complex Server (daemon, mod_perl and mod_soap)

Access to Remote Services

Access With Service Description (WSDL)

Security (SSL, basic/digest authentication, cookie-based authentication, ticket-based authentication, access control)

Handling LoLs (List of Lists, Structs, Objects, or something else)

More Complex Server (daemon, mod_perl and mod_soap)

You shouldn't have many problems with the CGI-based SOAP server you created in the first part of this article; however, performance could be significantly better. The next logical step might be to implement SOAP services using accelerators (like PerlEx or VelociGen) or persistent technologies (like mod_perl). Another lightweight solution might be to implement the SOAP service as an HTTP daemon; in that case, you don't need to use a separate Web server. This might be useful in a situation where a client application accepts SOAP calls, or for internal usage.

HTTP daemon
The following code shows an example implementation for a HTTP daemon:

4.a. server (HTTP daemon)

 #!perl -w

  use SOAP::Transport::HTTP;

  use Demo;

  # don't want to die on 'Broken pipe' or Ctrl-C
  $SIG{PIPE} = $SIG{INT} = 'IGNORE';

  $daemon = SOAP::Transport::HTTP::Daemon
    -> new (LocalPort => 80)
    -> dispatch_to('/home/soaplite/modules')
  ;

  print "Contact to SOAP server at ", $daemon->url, "\n";
  $daemon->handle;

Not much difference from the CGI server (Dynamic), huh? And it makes the same interface accessible, only through a different endpoint. This code is all you need to run the SOAP server on your computer without anything else.

HTTP daemon in VBScript
Similar code in VBScript may look like:

4.b. server (HTTP daemon, VBScript)

 call CreateObject("SOAP.Lite") _
   .server("SOAP::Transport::HTTP::Daemon", _
     "LocalPort", 80) _
   .dispatch_to("/home/soaplite/modules") _
   .handle

This is all you need to run SOAP server on a Microsoft platform (and it will run on Win9x/Me/NT/2K as soon as you register Lite.dll with regsvr32 Lite.dll).

ASP/VB
An ASP server could be created with VBScript or PerlScript code:

4.c. server (ASP server, VBScript)

 <%
    Response.ContentType = "text/xml"
    Response.Write(Server.CreateObject("SOAP.Lite") _
      .server("SOAP::Server") _ 
      .dispatch_to("/home/soaplite/modules") _
      .handle(Request.BinaryRead(Request.TotalBytes)) _
    )
  %>
Apache::Registry
One of the easiest ways to significantly speed up your CGI-based SOAP server is to wrap it with the mod_perl Apache::Registry module. You need to configure it in the httpd.conf file:

4.d. server (Apache::Registry, httpd.conf)

 Alias /mod_perl/ "/Apache/mod_perl/"
  <Location /mod_perl>
    SetHandler perl-script
    PerlHandler Apache::Registry
    PerlSendHeader On
    Options +ExecCGI
  </Location>

Put the CGI script soap.mod_cgi in the /Apache/mod_perl/ directory mentioned above:

4.d. server (Apache::Registry, soap.mod_cgi)

 #!perl -w

  use SOAP::Transport::HTTP;
  
  SOAP::Transport::HTTP::CGI
    -> dispatch_to('/home/soaplite/modules')
    -> handle
  ;
mod_perl
Let's consider a mod_perl-based server now. To run it you'll need to put the SOAP::Apache module (Apache.pm) in any directory in @INC:

4.e. server (mod_perl, Apache.pm)

 package SOAP::Apache;

  use SOAP::Transport::HTTP;
  
  my $server = SOAP::Transport::HTTP::Apache
    -> dispatch_to('/home/soaplite/modules') 
	
  sub handler { $server->handler(@_) }

  1;

Then modify your httpd.conf file:

4.e. server (mod_perl, httpd.conf)

 <Location /soap>
    SetHandler perl-script
    PerlHandler SOAP::Apache
  </Location>
mod_soap
mod_soap allows you to create a SOAP server by simply configuring the httpd.conf or .htaccess file.

4.f. server (mod_soap, httpd.conf)

 # directory-based access
  <Location /mod_soap>
    SetHandler perl-script
    PerlHandler Apache::SOAP
    PerlSetVar dispatch_to "/home/soaplite/modules"
    PerlSetVar options "compress_threshold => 10000"
  </Location>

  # file-based access
  <FilesMatch "\.soap$">
    SetHandler perl-script
    PerlHandler Apache::SOAP
    PerlSetVar dispatch_to "/home/soaplite/modules"
    PerlSetVar options "compress_threshold => 10000"
  </FilesMatch>

Directory-based access turns a directory into a SOAP endpoint. For example, you may point your request to http://localhost/mod_soap (there is no need to create this directory).

File-based access turns a file with a specified name (or mask) into a SOAP endpoint. For example, http://localhost/somewhere/endpoint.soap.

Alternatively, you may turn an existing directory into a SOAP server if you put an .htaccess file inside it:

4.g. server (mod_soap, .htaccess)

 SetHandler perl-script
  PerlHandler Apache::SOAP
  PerlSetVar dispatch_to "/home/soaplite/modules"
  PerlSetVar options "compress_threshold => 10000"

Access to Remote Services

It's time now to re-use what has already been done and to try to call some services available on the Internet. After all, the most interesting part of SOAP is interoperability between systems where the communicating parts are created in different languages, running on different platforms or in different environments, and are providing interfaces with service descriptions or documentation. XMethods.net can be a perfect starting point.

Name of state based on state's number (in alphabetical order)
Frontier implementation has a test server that returns the name of a state based on a number you provide. By default, SOAP::Lite generates a SOAPAction header with the structure of [URI]#[method]. Frontier, however, expects SOAPAction to be just the URI, so we have to use on_action to modify it. In our example, we specify on_action(sub { sprintf '"%s"', shift }), so the resulting SOAPAction will contain only the URI (and don't forget the double quotes there).

5.a. client

 #!perl -w
  
  use SOAP::Lite;

  # Frontier http://www.userland.com/  $s = SOAP::Lite 
    -> uri('/examples')
    -> on_action(sub { sprintf '"%s"', shift })
    -> proxy('http://superhonker.userland.com/')
  ;

  print $s->getStateName(SOAP::Data->name(statenum => 25))->result;

You should get the output:

5.a. result

Missouri


Paul Kulchenko is a featured speaker at the upcoming O'Reilly Open Source Convention in San Diego, CA, July 23 - 27, 2001. Take this opportunity to rub elbows with open source leaders while relaxing in the beautiful setting of the beach-front Sheraton San Diego Hotel and Marina. For more information, visit our conference home page. You can register online.



Whois
We will target services with different implementations. The following service is running on a Windows platform:

5.b. client

 #!perl -w

  use SOAP::Lite;

  # 4s4c (aka Simon's SOAP Server Services For COM) http://www.4s4c.com/  print SOAP::Lite 
    -> uri('http://www.pocketsoap.com/whois')
    -> proxy('http://soap.4s4c.com/whois/soap.asp')
    -> whois(SOAP::Data->name('name' => 'yahoo'))
    -> result;

Nothing fancy here; 'name' is the name of the field and 'yahoo' is the value. That should give you the output:

5.b. result

 The Data in Network Solutions' WHOIS database is provided by Network
  Solutions for information purposes, and to assist persons in obtaining
  information about or related to a domain-name registration record.
  Network Solutions does not guarantee its accuracy. By submitting a
  WHOIS query, you agree that you will use this Data only for lawful
  purposes and that, under no circumstances will you use this data to:
  (1) allow, enable or otherwise support the transmission of mass
  unsolicited, commercial advertising or solicitations via e-mail
  (spam); or  (2) enable high volume, automated, electronic processes
  that apply to Network Solutions (or its systems). Network Solutions
  reserves the right to modify these terms at any time. By submitting
  this query, you agree to abide by this policy.
  Yahoo (YAHOO-DOM)                                            YAHOO.COM
  Yahoo Inc. (YAHOO27-DOM)                                     YAHOO.ORG
  Yahoo! Inc. (YAHOO4-DOM)                                     YAHOO.NET

  To single out one record, look it up with "!xxx", where xxx is the
  handle, shown in parenthesis following the name, which comes first.
Book price based on ISBN
In many cases the SOAP interface is just a front end that requests information, parses the response, formats it and returns according to your request. It may not be doing that much, but it saves you time on the client side and fixes this interface, so you don't need to update it each time your service provider changes format or content. In addition, the major players are moving quickly toward XML; for example, Google already has an XML-based interface for its search engine. Here is the service that returns the price of a book given its ISBN:

5.c. client

 #!perl -w

  use SOAP::Lite;

  # Apache SOAP http://xml.apache.org/soap/ (running on XMethods.net)

  $s = SOAP::Lite                             
    -> uri('urn:xmethods-BNPriceCheck')                
    -> proxy('http://services.xmethods.net/soap/servlet/rpcrouter');

  my $isbn = '0596000278'; # Programming Perl, 3rd Edition
  print $s->getPrice(SOAP::Data->type(string => $isbn))->result;

Here is the result for 'Programming Perl, 3rd Edition':

5.c. result

 39.96

Note that we explicitly specified the type to be 'string', because an ISBN looks like number and will be serialized by default as an integer. However, the SOAP server we work with requires it to be a string.

Currency exchange rates
This service returns the value of one unit of country1's currency converted into country2's currency:

5.d. client

 #!perl -w

  use SOAP::Lite;

  # GLUE http://www.themindelectric.com/ (running on XMethods.net)

  my $s = SOAP::Lite                             
    -> uri('urn:xmethods-CurrencyExchange')                
    -> proxy('http://services.xmethods.net/soap');

  my $r = $s->getRate(SOAP::Data->name(country1 => 'England'), 
                      SOAP::Data->name(country2 => 'Japan'))
            ->result;
  print "Currency rate for England/Japan is $r\n";

Which gives you (as of 2001/03/11):

5.d. result

 Currency rate for England/Japan is 175.4608
NASDAQ quotes
This service returns a delayed stock quote based on a stock symbol:

5.e. client

 #!perl -w

  use SOAP::Lite;

  # GLUE http://www.themindelectric.com/ (running on XMethods.net)

  my $s = SOAP::Lite                             
    -> uri('urn:xmethods-delayed-quotes')                
    -> proxy('http://services.xmethods.net/soap');

  my $symbol = 'AMZN';
  my $r = $s->getQuote($symbol)->result;
  print "Quote for $symbol symbol is $r\n";

It may (or may not, depending on how Amazon is doing) give you:

5.e. result

 Quote for AMZN symbol is 12.25

Access with service description (WSDL)

Although support for WSDL 1.1 is limited in SOAP::Lite for now (the service description may work in some cases, but hasn't been extensively tested), you can access services that don't have complex types in their description:

6.a. client

 #!perl -w

  use SOAP::Lite;
  
  print SOAP::Lite
    -> service('http://www.xmethods.net/sd/StockQuoteService.wsdl')
    -> getQuote('MSFT');

If we take a look under the hood we'll find that SOAP::Lite requests a service description, parses it, builds the stub (a local object that has the same methods as the remote service) and returns it to you. As a result, you can run several requests using the same service description:

6.b. client

 #!perl -w

  use SOAP::Lite;
  
my $service = SOAP::Lite -> service('http://www.xmethods.net/sd/StockQuoteService.wsdl');
print 'MSFT + ORCL = ', $service->getQuote('MSFT') + $service->getQuote('ORCL');

The service description doesn't need to be on the Internet; you can access it from your local drive also:

6.c. client

 #!perl -w

  use SOAP::Lite
    service => 'http://www.xmethods.net/sd/StockQuoteService.wsdl',
    # service => 'file:/your/local/path/StockQuoteService.wsdl',
    # service => 'file:./StockQuoteService.wsdl',
  ;

  print getQuote('MSFT'), "\n";

This code works similar to the previous example (in OO style), but loads the description and imports all the methods, so you can use the functional interface.

And finally, a couple of one-liners for those who like to do something short and simple (albeit useful and powerful):

6.d. client

 # The following command is split for readability
  perl "-MSOAP::Lite service=>'http://www.xmethods.net/sd/StockQuoteService.wsdl'" 
       -le "print getQuote('MSFT')"

  perl "-MSOAP::Lite service=>'file:./quote.wsdl'" -le "print getQuote('MSFT')"

The last example (marked line) seems to be the shortest SOAP method invocation.

Security (SSL, basic/digest authentication, cookie-based authentication, ticket-based authentication, access control)

Though SOAP doesn't impose any security mechanisms (unless you count the SOAP Security Extensions: Digital Signature specification), the extensibility of the protocol allows you to leverage many security methods that are available for different protocols, like SSL over HTTP or S/MIME. We'll consider how SOAP can be used together with SSL, basic authentication, cookie-based authorization and access control.

SSL
Let's start with SSL. Surprisingly there is nothing SOAP-specific you need to do on the server side, and there is only a minor modification on the client side: just specify https: instead of http: as the protocol for your endpoint and everything else will be done for you. Obviously, both endpoints should support this functionality and the server should be properly configured.

7.a. client

 #!perl -w
  
  use SOAP::Lite +autodispatch => 
    uri => 'http://www.soaplite.com/My/Examples',

    proxy => 'https://localhost/cgi-bin/soap.cgi',

    on_fault => sub { my($soap, $res) = @_; 
      die ref $res ? $res->faultdetail : $soap->transport->status, "\n";
    }
  ;

  print getStateName(21);
Basic authentication
The situation gets even more interesting with authentication. Consider this code that accesses an endpoint that requires authentication.

7.b. client

 #!perl -w

  use SOAP::Lite +autodispatch => 
    uri => 'http://www.soaplite.com/My/Examples', 
    proxy => 'http://services.soaplite.com/auth/examples.cgi', 
    on_fault => sub { my($soap, $res) = @_; 
      die ref $res ? $res->faultdetail : $soap->transport->status, "\n";
    }
  ;

  print getStateName(21);

Keep in mind that the password will be in clear text during the transfer (not exactly in clear text; it will be base64 encoded, but that's almost the same) unless the user uses https (i.e. authentication doesn't mean encryption).

The server configuration for an Apache Web server with authentication can be specified in a .conf or in .htaccess file, and may look like this:

7.b. server (.htaccess)

 AuthUserFile /path/to/users/file/created/with/htpasswd
  AuthType Basic
  AuthName "SOAP::Lite authentication tests"
  require valid-user

If you run example 7.b against this endpoint, you'll probably get the following error:

7.b. result

 401 Authorization Required

You may provide the required credentials on the client side (user soaplite, and password authtest) overriding the function get_basic_credentials() in the class SOAP::Transport::HTTP::Client:

7.c. client

 #!perl -w
  
  use SOAP::Lite +autodispatch => 
    uri => 'http://www.soaplite.com/My/Examples', 
    proxy => 'http://services.soaplite.com/auth/examples.cgi', 
    on_fault => sub { my($soap, $res) = @_; 
      die ref $res ? $res->faultdetail : $soap->transport->status, "\n";
    }
  ;

  sub SOAP::Transport::HTTP::Client::get_basic_credentials { 
    return 'soaplite' => 'authtest';
  }

  print getStateName(21);

That gives you the correct result:

7.c. result

Massachusetts

Alternatively you may provide this information with a credentials() functions, but you need to specify the host and realm also:

7.d. client

 #!perl -w

  use SOAP::Lite +autodispatch => 
    uri => 'http://www.soaplite.com/My/Examples',

    proxy => [
      'http://services.soaplite.com/auth/examples.cgi', 
      credentials => [
        'services.soaplite.com:80',        # host:port
        'SOAP::Lite authentication tests', # realm
        'soaplite' => 'authtest',          # user, password
      ]
    ],

    on_fault => sub { my($soap, $res) = @_; 
      die ref $res ? $res->faultdetail : $soap->transport->status, "\n";
    }
  ;

  print SOAP->getStateName(21);

Under modern Perl you may get a warning about ``deprecated usage of inherited AUTOLOAD''. To avoid it use the full syntax: SOAP->getStateName(21) instead of getStateName(21).

The simplest and most convenient way would probably be to provide the user and password embedded in a URL. Surprisingly, this works:

7.e. client

 #!perl -w

  use SOAP::Lite;

  print SOAP::Lite
    -> uri('http://www.soaplite.com/My/Examples')

    -> proxy('http://soaplite:authtest@services.soaplite.com/auth/examples.cgi')

    -> getStateName(21)
    -> result;
Cookie-based authentication
Cookie-based authentication also doesn't require much work on the client side. Usually, it means that you need to provide credentials in some way, and if everything is OK, the server will return a cookie on success, and will then check it for all subsequent requests. Using available functionality you may not only support this behavior on the client side in one session, but even store cookies in a file and use the same server session for several runs. All you need to do is:

7.f. client

 #!perl -w

  use SOAP::Lite; 
  use HTTP::Cookies;

  my $soap = SOAP::Lite
    -> uri('urn:xmethodsInterop')

    -> proxy('http://services.xmethods.net/soap/servlet/rpcrouter', 
             cookie_jar => HTTP::Cookies->new(ignore_discard => 1));

  print $soap->echoString('Hello')->result;

All the magic is in the cookie jar. You may even add or delete cookies between calls, but the underlying module does everything you need by default. Add the option file => 'filename' to the call to new() to save and restore cookies between sessions. Not much work, huh? Kudos to Gisle Aas on that!

Ticket-based authentication
Ticket-based authentication is a bit more complex. The logic is similar to cookie-based authentication, but it is executed at the application level, instead of at the transport level. The advantage is that it works for any SOAP transport (not only for HTTP) and gives you a bit more flexibility. As a result, you won't get support from the Web server and you'll have to do everything manually. No big deal, right?

The first step is the ticket generation. We'll build a ticket that contains an e-mail address, a time and a signature.

7.g. server (TicketAuth)

 package TicketAuth;

  # we will need to manage Header information to get a ticket
  @TicketAuth::ISA = qw(SOAP::Server::Parameters);

  # ----------------------------------------------------------------------
  # private functions
  # ----------------------------------------------------------------------

  use Digest::MD5 qw(md5);

  my $calculateAuthInfo = sub {
    return md5(join '', 'something unique for your implementation', @_);
  };

  my $checkAuthInfo = sub {
    my $authInfo = shift;
    my $signature = $calculateAuthInfo->(@{$authInfo}{qw(email time)});
    die "Authentication information is not valid\n" 
      if $signature ne $authInfo->{signature};
    die "Authentication information is expired\n" 
      if time() > $authInfo->{time};
    return $authInfo->{email};
  };

  my $makeAuthInfo = sub {
    my $email = shift;
    my $time = time()+20*60; # signature will be valid for 20 minutes
    my $signature = $calculateAuthInfo->($email, $time);
    return +{time => $time, email => $email, signature => $signature};
  };

  # ----------------------------------------------------------------------
  # public functions
  # ----------------------------------------------------------------------

  sub login { 
    my $self = shift;

    pop; # last parameter is envelope, don't count it
    die "Wrong parameter(s): login(email, password)\n" unless @_ == 2;
    my($email, $password) = @_;

    # check credentials, write your own is_valid() function
    die "Credentials are wrong\n" unless is_valid($email, $password);

    # create and return ticket if everything is ok
    return $makeAuthInfo->($email);

  }

  sub protected { 
    my $self = shift;

    # authInfo is passed inside the header
    my $email = $checkAuthInfo->(pop->valueof('//authInfo'));

    # do something, user is already authenticated 
    return;
  }

It would be very careless (and insecure) to create calculateAuthInfo() as a normal, exposed function, because a client could invoke it directly and generate a valid ticket without providing valid credentials (unless you forbid it in the SOAP server configuration, but we'll show another way). Therefore, we create calculateAuthInfo(), checkAuthInfo() and makeAuthInfo() as 'private' functions, so only other functions inside the same file can access it. It effectively prevents clients from accessing them directly.

The login() function returns a hash that has an e-mail and time inside, as well as an MD5 signature that prevents the user from altering this information. Since the server used a secret string during signature generation, the user is not able to tamper with the resulting signature. To access protected methods, the client has to provide the obtained ticket in the header:

7.g. fragment

 # login
  my $authInfo = login(email => 'password');

  # convert it into the Header
  $authInfo = SOAP::Header->name(authInfo => $authInfo);

  # invoke protected method
  protected($authInfo, 'parameters');

This is just a fragment, but it should give you some ideas on how to implement ticket-based authentication on application level. You could even get the ticket in one place (via HTTP for example) and access a SOAP server via SMTP providing this ticket (ideally you should use PKI [public key infrastructure] for that matter).

Access control
Why would you need access control? Imagine you have a class and want to give access to it selectively; for example, read access to one person and read/write access to another person or a list of people. At a low level, read and write access means access to specific functions/methods in class.

You could put this check in at the application level (for example with ticket-based authentication), or you could split your class into two different classes and give one person access only to one of them. Neither of these is optimal solutions. We consider a different approach, where you create two different endpoints that refer to the same class on the server side, but have different access options.

7.e. server (first endpoint)

 use SOAP::Transport::HTTP;
  
  use Protected;
  SOAP::Transport::HTTP::CGI
    -> dispatch_to('Protected::readonly')
    -> handle
  ;

This endpoint will have access only to readonly() method in Protected class.

7.e. server (second endpoint)

 use SOAP::Transport::HTTP;

  use Protected;
  SOAP::Transport::HTTP::CGI
    -> dispatch_to('Protected')
    -> handle
  ;

This endpoint will have unrestricted access to all methods/functions in Protected class. Now you can put it under basic, digest or some other kind of authentication to prevent unauthorized access.

Thus, by combining the capabilities of a Web server with the SOAP server you can create an application that best suites your needs.

Handling LoLs (List of Lists, Structs, Objects, or something else)

Processing complex data structures isn't different in any aspect from the usual processing in your programming language. The general rule is simple: 'Treat the result of a SOAP call as a variable of specified type'.

The next example shows a service that works with array of structs:

8.a. client

 #!perl -w

  use SOAP::Lite;

  my $result = SOAP::Lite
        -> uri('urn:xmethodsServicesManager')
        -> proxy('http://www.xmethods.net/soap/servlet/rpcrouter')
        -> getAllSOAPServices();

  if ($result->fault) {
    print $result->faultcode, " ", $result->faultstring, "\n";
  } else {
    # reference to array of structs is returned
    my @listings = @{$result->result};

    # @listings is the array of structs
    foreach my $listing (@listings) {
      print "-----------------------------------------\n";
      # print description for every listing
      foreach my $key (keys %{$listing}) {
        print $key, ": ", $listing->{$key} || '', "\n";
      }        
    }
  }

The same is true about structs inside other structs, lists of objects, objects that have lists inside, etc. 'What you return on server side is what you get on client side, and let me know if you get something else.'

(OK, not always. You MAY get a blessed array even when you return a simple array on the other side and you MAY get a blessed hash when you return a simple one, but it won't change anything in your code, just access it as you usually do).


Major contributors:

Nathan Torkington
Basically started this work and pushed the whole process.

Tony Hong
Invaluable comments, fixes and input help me keep this material correct, fresh and simple.

Visit the home of the Perl programming language: Perl.org

Sponsored by

Monthly Archives

Powered by Movable Type 5.13-en