O'Reilly Open Source Convention
oreilly.comO'Reilly Network
Conferences International Safari: Books Online

Arrow Home
Arrow Registration
Arrow Speakers
Arrow Keynotes
Arrow Tutorials
Arrow Sessions
Arrow At-a-Glance
Arrow BOFs
Arrow Events
Arrow Exhibitors
Arrow Sponsors
Arrow Hotel/Travel
Arrow Venue Map
Arrow See & Do
Arrow Press
Arrow Mail List
Kids World

From the Frontiers of Research to the Heart of the Enterprise
O'Reilly Open Source Convention
Sheraton San Diego Hotel and Marina
July 22-26, 2002 -- San Diego, CA



Programming with Iterators and Generators
Mark-Jason Dominus, Plover Systems Co.

Track: Perl
Date: Tuesday, July 23
Time: 8:45am - 12:15pm
Location: Marina VI

Sometimes you'll write a function that takes too long to run because it produces too much useful information. A search function might locate ten thousand matches. A database query might return ten million records. What can you do?

Perl provides a simple and familiar model for dealing with such problems: The filehandle! Instead of reading every file in one giant gulp, we can use filehandles to staunch the flow of information, trickling it into the program in manageable gulps.

In this class, we'll see several important modules, such as File::Find and DBI, which use this approach. We'll go inside these modules and see how to implement filehandle-like data structures and objects ourselves. We'll see how to write functions that suspend themselves and then pick up later where they left off. We'll learn how to take long-running slow functions and convert them into speedy filehandle-like data generator objects.

This fundamental technique is a mainstay of programmers in other languages, but isn't as well-known as it should be. You'll be amazed at how many difficult problems become simple when you unleash the power of filehandles.

  • Introduction:
    • Filehandles are Iterators
    • Iterators are Objects
    • Common Examples of Iterators: readdir, each, and DBI
  • Homemade iterators
  • Examples
    • File tree scanning
    • Permutations
    • Genomic Sequence Generator
    • Filehandle Iterators
    • A Flat-File Database
    • Searching Databases Backwards
    • Random Number Generation
  • Alternatives to Iterators
  • Filters and Transforms: 'map' and 'grep' for iterators
  • Recursively-Constructed Iterators
  • The Semipredicate Problem
  • Alternative Interfaces to Iterators
  • An Extended Example: Web Spiders
  • Converting slow recursive functions to iterators

oreilly.com Home | O'Reilly Bookstores | How to Order | O'Reilly Contacts
International | About O'Reilly | Affiliated Companies | Privacy Policy

© 2001, O'Reilly Media, Inc.