Jim White's Blog

November 23, 2013 12:21 AM
Bundling Local JARs for your Groovy scripts the Grape Way
A FAQ which often does not receive a satisfactory answer is how to add local JAR files to a Groovy script without having to fiddle with the dreaded CLASSPATH environment variable. Folks see that things are quite nice when using Grape with a Maven repository and wish they could do the same with local JAR files. When creating IFCX Wings I implemented Groovy For OpenOffice (pre-Grape) which also used Apache Ivy for dynamically extending the classpath. One difference though is that I directly supported including JAR files which are not in a repository but are simply referenced by URL, whether local (file) or remote (http/https). It turns out though that Grape can be used with local JAR files in fairly nice fashion by keeping them in a Maven-format repository in the local filesystem. Here's how to do that in pretty friendly manner.

Use Maven to put the JAR files in a Local Repository

Maven will install files in a local repository for you without fussing around with a POM file. We'll use the mvn deploy-file command and a local file URL that is relative to where our script is located which will be file:repo in this example. Notice that the file URL has no leading slash and so is relative to the working directory. Naturally you'll need to use an appropriate groupId and artifactId (please be a good netizen and use a groupId that the reverse of a domain name that is assigned to you or your organization). For this example I'm demonstrating a little script that uses Apache Lucene and I'll use my groupId so that Grape will only use my copy. One flaw in this scheme at the moment is that transitive dependencies are not managed although it should be possible to work that out and I'll look into it later. The upshot of that is we have to list each JAR file individually, which shouldn't really be an issue since we're already just wrangling JAR files by hand anyhow.

Layout of my Working Directory

$ find *
lib
lib/lucene-analyzers-common-4.5.1.jar
lib/lucene-core-4.5.1.jar
whats_new.groovy

Top of my Groovy Script

#!/usr/bin/env groovy

@GrabResolver(name='script', root='file:repo')
@Grab(group='org.ifcx.lucene', module='lucene-core', version='4.5.1')
@Grab(group='org.ifcx.lucene', module='lucene-analyzers-common', version='4.5.1')

import org.apache.lucene.analysis.standard.StandardAnalyzer
import org.apache.lucene.document.Document
import org.apache.lucene.document.Field
import org.apache.lucene.index.DirectoryReader
...
The full script (using the default Maven Central repo) can be see in this gist https://gist.github.com/jimwhite/7609525.

Create the Local Maven Repository

We use the command mvn deploy:deploy-file -Durl=file:repo -Dfile=/path/to/jar -DgroupId=name -DartifactId=name -Dversion=ver to copy the JAR file to a new Maven repository in the local directory repo.
$ mvn deploy:deploy-file -Durl=file:repo -Dfile=lib/lucene-core-4.5.1.jar -DgroupId=org.ifcx.lucene -DartifactId=lucene-core -Dpackaging=jar -Dversion=4.5.1
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-deploy-plugin:2.7:deploy-file (default-cli) @ standalone-pom ---
Uploading: file:repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar
Uploaded: file:repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar (2244 KB at 16025.9 KB/sec)
Uploading: file:repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom
Uploaded: file:repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom (399 B at 389.6 KB/sec)
Downloading: file:repo/org/ifcx/lucene/lucene-core/maven-metadata.xml
Uploading: file:repo/org/ifcx/lucene/lucene-core/maven-metadata.xml
Uploaded: file:repo/org/ifcx/lucene/lucene-core/maven-metadata.xml (306 B at 298.8 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.820s
[INFO] Finished at: Sat Nov 23 00:42:08 PST 2013
[INFO] Final Memory: 3M/81M
[INFO] ------------------------------------------------------------------------

$ mvn deploy:deploy-file -Durl=file:repo -Dfile=lib/lucene-analyzers-common-4.5.1.jar -DgroupId=org.ifcx.lucene -DartifactId=lucene-analyzers-common -Dpackaging=jar -Dversion=4.5.1
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-deploy-plugin:2.7:deploy-file (default-cli) @ standalone-pom ---
Uploading: file:repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar
Uploaded: file:repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar (1549 KB at 12196.3 KB/sec)
Uploading: file:repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom
Uploaded: file:repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom (411 B at 401.4 KB/sec)
Downloading: file:repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml
Uploading: file:repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml
Uploaded: file:repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml (318 B at 155.3 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.745s
[INFO] Finished at: Sat Nov 23 00:42:28 PST 2013
[INFO] Final Memory: 3M/81M
[INFO] ------------------------------------------------------------------------

A New Repository is Born

$ find *
lib
lib/lucene-analyzers-common-4.5.1.jar
lib/lucene-core-4.5.1.jar
repo
repo/org
repo/org/ifcx
repo/org/ifcx/lucene
repo/org/ifcx/lucene/lucene-analyzers-common
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar.md5
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.jar.sha1
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom.md5
repo/org/ifcx/lucene/lucene-analyzers-common/4.5.1/lucene-analyzers-common-4.5.1.pom.sha1
repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml
repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml.md5
repo/org/ifcx/lucene/lucene-analyzers-common/maven-metadata.xml.sha1
repo/org/ifcx/lucene/lucene-core
repo/org/ifcx/lucene/lucene-core/4.5.1
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar.md5
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.jar.sha1
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom.md5
repo/org/ifcx/lucene/lucene-core/4.5.1/lucene-core-4.5.1.pom.sha1
repo/org/ifcx/lucene/lucene-core/maven-metadata.xml
repo/org/ifcx/lucene/lucene-core/maven-metadata.xml.md5
repo/org/ifcx/lucene/lucene-core/maven-metadata.xml.sha1
whats_new.groovy

We're Ready to Rock

$ ./whats_new.groovy index_dir ~/Downloads/ucsd_2.0_content/editor_source/command.text 
SEGMENT PROCEDURE EDITCORE;

(* Core procedures.  Execute these commands until either a set environment
   comes along or a quit command. *)
...
There are 263 new lines of 263 total.

Note that the script was run with the working directory being the one containing the Groovy script. That is essential because the root URL we used in the @GrabResolver annotation is relative to the working directory. That isn't quite as inconvenient as it might seem since the URL will only be used if the artifacts are not in the Groovy Grapes cache (~/.groovy/grapes), which typically will only be the first time the script is run by a user. The caching though does mean that if the JAR changes then the version number needs to change and the script must again be run from the working directory.

Update

As a result of addressing this issue I've created an improvement for Grape that makes it work really nicely with these Maven repositories collocated with the Groovy scripts that use them. Follow GROOVY-6451 to see when this feature is ready to use. I'll also post here as well of course.

Go Forth in Groovy Style

That's all there is to it. The working directory can be delivered using your favorite method such as tar and ssh or what-have-you. The lib directory doesn't need to be included of course and is only shown here for completeness - the path given to mvn deploy:deploy-file could easily have been elsewhere (including your ~/.m2 or ~/.groovy/grapes cache).
By Jim White  Permalink -Comments
December 22, 2008 1:04 AM
Bottom of the Ninth, or, Just how "Open" is Java anyways?
A recent post on the OCJUG list asked to what degree is Java "open" and when will everyone agree that it is "open" to their satisfaction.

The fact that "Java" is a trademark (the significance of which was made obvious when Sun changed their stock symbol from SUNW to JAVA) means that what most folks think Java is will never be as open or free as some folks want.

Furthermore, Sun has a dual license for their Java distribution which includes OpenJDK. So if you want to contribute to the most popular JDK distribution, you have to agree that Sun can also use your efforts under terms that are not necessarily free or open (although they do promise to always make your contribution available under a FSF and/or OSI approved license).

http://openjdk.java.net/contribute/

So OpenJDK has two strikes against it.

Here comes the next pitch, aaannd OpenJDK is GPL licensed, a home run!

I was amazed and pleased that Sun went all the way and used GPL for Java. After all, I'd been saying that Java would eventually be OSS, even after the ISO talk was replaced by the JCP.

That means this pillar of the Great Java Renaissance of 2006 is strong as it possibly can be. One of those strengths is that anyone who isn't satisfied with the degree of openness can simply fork with total abandon and with the best license for software freedom. That is a reality and one of the first projects spawned by OpenJDK is IcedTea, which is a free and open implementation of Java.

As for folks saying Java being OSS doesn't matter. They are quite mistaken. Already we've seen significant developments such as SoyLatte which counters Apple's weak support of Java, and research work on Java being truly free and open in the Da Vinci Machine Project. This change in research work is important because in the past such work, if done with Sun's JDK, used a license that meant the resulting code rarely ever left the university.

That the OpenJDK is an effective OSS project is clear because all of these efforts are now part of the OpenJDK project itself, rather than forking or otherwise choosing to stand alone or with another group.

That is only the beginning and, by being real Open Source Software, we can rest assured that Java will grow in strength over the next decade just as it did the first. Of course we can also be assured that there are plenty of folks that will disagree with just about every aspect of all this jazz.

By Jim White  Permalink -Comments
December 12, 2008 10:58 AM
Grid computing with Groovy
Alex Tkachman posted a question about Groovy style for checkpointed calculations.

Here's my first try:

// Proof of concept for Groovy checkpointed calculation with closures.
// @author Jim White <jim@pagemsiths.com>
// http://www.ifcx.org/

Session session = new Session()

// def initState = new Expando(data:'datalocation')
def initState = [data:'datalocation']

def endState = eachWithCheckpoints(session, 'state1', initState, [
      { stepCount = longInitialConditionsCalculation(data) }
    , { accumulator = 0
        iterateWithCheckpoints(session, 'state2', it.state
           , 0..stepCount
           , { accumulator += 2 * it.step })
      }
    , { result = "We did $stepCount iterations and the answer is $accumulator!" }
])

def longInitialConditionsCalculation(d) { 10 }

println endState.result

class Session {
    Object loadCheckpoint(String k) { null }
    void saveCheckpoint(String k, Serializable s) { }
}

def eachWithCheckpoints(Session session, String key, def initialState, List<Closure> closures) {
    def control = session.loadCheckpoint(key)

    if (control.is(null)) {
        control = [step:0, state:initialState]
    }

    while (control.step < closures.size()) {
        Closure clos = closures[control.step]

        clos.delegate = control.state
        clos.resolveStrategy = Closure.DELEGATE_FIRST

        clos.call(control)

        control.step += 1

        session.saveCheckpoint(key, control)
    }

    control.state
}

def iterateWithCheckpoints(Session session, String key, def initialState, Range range, Closure clos) {
    def control = session.loadCheckpoint(key)

    if (control.is(null)) {
        // Leave reversed range case as an exercise for the reader...
        assert !range.isReverse()

        control = [step:range.from, state:initialState]
    }

    clos.delegate = control.state
    clos.resolveStrategy = Closure.DELEGATE_FIRST

    while (control.step <= range.to) {
        clos.call(control)

        control.step += 1

        session.saveCheckpoint(key, control)
    }

    control.state
}
==>
We did 10 iterations and the answer is 110!

The idea is either you're doing a sequence of different steps or iterating the same step some number of times. Notice that the state can be any serializable thing. Map or Expando is handy, but some bean or other class would be fine.

Obviously there are further refinements possible such as reversed and stepped ranges, and also a more functional style is possible. Either a builder or controller class would streamline things a bit by hiding some of the details of the eachWithCheckpoints/iterateWithCheckpoints calls.

If Groovy had serializable iterators for the control step, that would make iterateWithCheckpoints nicer and let it change from taking just a simple integer range to a list.

Naturally where there this is headed is cloud-powered click-and-it-goes Wings via your web browser using OOHTML.

By Jim White  Permalink -Comments
December 9, 2008 5:25 PM
TREE-META
I've been interested in TREE-META since I discovered it while at UC Irvine. We used a version for the UCSD p-System on the Teraks (desktop LSI-11/23). It had a pleasant and compact syntax that was more convenient than the LISP that took it's place for me.

For several years I've been on the lookout for information on TREE-META but haven't been able to find out a great deal. After about the third time listening to The Mother of All Demos I heard them mention that TREE-META was the language used to implement the "special purpose languages", which we call "domain-specific" today.

I've been unable to locate a copy of any version the program or the tech report describing the language, although I've seen hints than some folks have used it fairly recently.

In commemoration of the 40th anniversary of the MoAD (also at Wired), I've created a Wikipedia stub about TREE-META as well as one here for non-WP suitable material. The WP article is very rough, but I'm hoping that others with information will bring it forward and collect it there.

By Jim White  Permalink -Comments
December 21, 2007 1:16 PM
OLPC XO Laptop : Give One Get One Now!
Got my G1G1 XO Laptop yesterday evening and have been fiddling with it for it hours. I'm very pleased and impressed with the design and believe that the XO will deliver on OLPC's vision of bringing computer literacy to, and unleashing creativity in, the children of the developing world. Actually I'm convinced it will do that for more folks than just them!

I did take some pictures of course, but I'll point you to some videos that do a better job of introducing the XO:

Sugar is the XO's desktop GUI. It is very good and has some excellent innovations. Three of the elements that stand out are visual map of the mesh network neighborhood, visual map of running applications (the "activities" in the circle are running, as opposed to the menu bar on the bottom which are those that can be launched), and, most importantly IMO, the activity-centric Journal (rather than the expired files-and-folders desktop metaphor). Having a task-oriented organization scheme is something I've been wanting/expecting in Mac OS for twenty years, perhaps now it'll happen when the Apple folks see they've been scooped in UI design by a Linux machine with 128MB of RAM (it's 1984 all over again!). Of course Genius Folders will be a dandy enhancement to the Journal's tagging scheme.

For young children the XO is probably darn near perfect. The built-in tools (writing, music, chat, web browser, video, graphics, data recorder, and calculator) and games are the obvious starting point. For real computer powered creativity it has an impressive set of easy-to-use programming tools. Python is the XO's primary scripting language and the Pippy and Develop (Activity-building Activity) IDEs are a natural progression from EToys (a LOGO-like environment in Squeak/Smalltalk) and Turtle Art visual programming activities.

Some obvious low-hanging fruit for making the XO a vehicle for teaching older children and young adults is packaging eBooks and on-line course material from sources like Project Guttenberg, MIT OpenCourseWare, and Stanford on iTunes U. I see a flourishing library system of SD cards and USB memories. Naturally the same materials would work dandy on the Sony and Amazon eBook readers. Somewhat more challenging (but of personal interest to me) is a web newsfeed system to replace my newspaper subscription.

Java-oriented folks will of course be disappointed that XO doesn't normally support Java, but being a resource-constrained platform that is pretty much unavoidable (Microsoft has been making the same whine wrt Windows on XO). While there has been some work done to make Java available on XO including a JNLP handler, I think a more useful approach would be Google Android for XO. And I'm sure Gosling would agree since it is his opinion that the PC for the developing world is the cell phone and that the OLPC Project is bad idea.

Another idea for developers I have is porting Sugar to the Nokia N800/N810 which have specifications that are similar to the XO (actually they're a bit slower and ARM-powered but the memory and display sizes are quite close). There is some discussion about Sugar and Nokia's Maemo but there doesn't seem to be any indication of a port in progress.

There is still time to participate in Give One Get One as the deadline was extended to December 31st, so if you haven't ordered yours yet please do it now! Not only do you get a tax deduction for the donated XO, you also get a year of T-Mobile WiFi which covers a lot of places including a zillion Starbucks.

When you do get your XO Laptop, please let me know as I'm interested in meeting up with other OLPC-minded folks.

By Jim White  Permalink -Comments
December 28, 2006 12:47 PM
Free Music
I've finally written up my wish for the glorious mash-up of Google Audio and Music To Go that will bring Free Music for everybody!
By Jim White  Permalink -Comments
June 27, 2006 12:22 PM
Solar Unicat
A self-contained hydrogen fuel system in a Unicat would be ultracool. The biggest, baddest Green Thing on six wheels!

Off-the-shelf components are available to do it with moderate cost:

Extreme Green RV Schematic

The Feds are on the hydrogen bandwagon: http://hydrogen.energy.gov/

Honda is moving the FCX to production: http://www.greencarcongress.com/2005/10/hondas_more_pow.html

Some high school students in Arizona made a solar hydrogen truck for $10K: http://www.fuelcellsworks.com/Supppage1259.html

Another news page: http://freeenergynews.com/Directory/Hydrogen/index.html


Green RV

I need to set up a pahe & category for Green RV technology. But in the meantime I'll comment on the Solar Unicat post.

This AutoBlogGreen item illustrates movement in the right direction: Ford Airstream concept: a shiny, hydrogen-powered PHEV funmobile. Not nearly as exciting as the Solar Unicat extreme Green RV concept, but a clear indication we're getting there.

--Jim White, 26-Jan-2007

By Jim White  Permalink -Comments
June 24, 2006 9:26 AM
Huh?

What happened to the listing for the other entry?

Can we have nested/Wiki Page in a Folder(info)?

No. Those are treated as attachments.

By Jim White  Permalink -Comments
June 17, 2006 12:47 AM
Hello World!
The IFCX : Internet Foundation Classes eXtreme! web site is alive!

I've been working on implementing Wiki Publishing for this web site (based on JSPWiki), and so haven't put any content here yet. Over the next several weeks I'll be filling in some details as I prepare for my Presentation Introducing IFCX at the July LAJUG .

If you're the sort who just can't wait to find out what this all about, you can check out my little talk at the end of last week's OCJUG meeting on Google Video. My intention was to give a little 10 to 15 minute sneak peek on GWT and the start of an IFCX Builder for it. That's basically what I did, but it ran rather long...

Check out the next entry for the announcement with the details.

By Jim White  Permalink -Comments