Finding duplicate records in a books to prisoners database application

High on my list of neglected tech. projects is the Testament books to prisoners database web application.  This is the database program that projects like the Midwest Pages to Prisoners Project use to track packages sent and returned and books requested in the hopes avoiding delays in delivering books to incarcerated people and to provide metrics that grant providers like.

One of the design challenges has to do with duplicate records.  Recipients of books are identified by their state/federal department of correction (DOC) number (if they’re in a state or federal prison – most jails don’t use ID numbers), their state of incarceration and their name.  I assume that the database was designed originally to minimize barriers for the book project volunteers so both the name and DOC# are free text fields.  Javascript is used to match existing records based on the DOC#, but there is still a large possibility for duplicate records.

The reason for duplicate records is that both the person writing to request books and the volunteer may list their name and/or DOC# inconsistently.  For instance, the state may store the DOC# in their database as A-123456 but the incarcerated person may write it as A123456 A-123-456 or just 123456.  Volunteers who don’t know about this and aren’t careful may not check beforehand for an existing record.

This is probably preventable through more sophisticated validation, but we still need a way to find duplicates in the existing records.  As this application is written in the Django framework, I want to try to use the Django API to find matches.

At first thought, it seems like I will have to iterate through each inmate record and check if there is a duplicate record.  This seems pretty slow, but I can’t think of a better way to do this.  At this point, there aren’t so many records that this approach will fail, but it would be nice to do something slicker.

The other problem is how to match a duplicate.  One approach might be to build a regexp for the DOC# (for instance, match either the first character or omit it, allow dashes or spaces between all characters, …) and then use the iregexp field lookup to try to find matches. One challenge with this is that the current Testament codebase is using Django 0.97 (I think) and iregexp is only available starting in 1.0.  Maybe it’s time we updated our code anyway.

There is also the Python difflib module that can compute deltas between strings.  However, it seems like this would slow things down even further because you would have to load each inmate object and then use difflib to compare the DOC#s.  I assume that the previous approach would be faster because the regexp matching happens at the database level.

Django: Querying data from the Python shell

I needed to get some stats for some research that we’re doing and was happy to see that you can use Django and the python shell to query testament data in a way that’s database independent.  It’s a little unintuitive if you’re thinking in SQL mode, but it is usable and super-helpful.  I wanted to share it with ya’ll in case you needed to quickly pull stats or examine info.

Helpful reference Django docs:

  • http://docs.djangoproject.com/en/dev/topics/db/queries/#making-queries
  • http://docs.djangoproject.com/en/dev/ref/models/querysets/#queryset-api-reference

Print the prison name, city, and state of all prisons that received a package sent by the Midwest Pages to Prisones Project from 2009-01-01 to 2009-03-22

geoff@btp:/var/www/testament/testament_trunk/btp$ python manage.py shell
>>> import datetime
>>> from core.models import Prison, Package
>>> start_date = datetime.date(2009, 1, 1)
>>> end_date = datetime.date(2009, 3, 22)
>>> prisons = Prison.objects.filter(package__sent_on__range=(start_date, end_date), package__group__username__exact='mwpp').distinct()
>>> for prison in prisons:
>>>    print "%s %s, %s" % (prison.name, prison.city, prison.state)

Join the ninth annual CALLS FROM HOME radio broadcast for prisoners.

The United States has 2.4 million people behind bars. Thousand Kites wants you to lend your voice to a powerful grassroots radio broadcast that reaches into our nation’s prison and lets those inside know they are not forgotten.
[]

We are asking you to call our toll-free line 877-518-0606 and speak directly to those behind bars this holiday season. (An answering machine will record your message) Read a poem, sing a song, or just speak directly from you heart. Speak to someone you know or to everyone—make it uplifting.

So call right now at 877-518-0606. We will post each call on our website as it comes in! Check our website www.thousandkites.org  to listen to your call and others!

CALLS FROM HOME will broadcast on over 200 radio stations across the country and be available for download from our website on December 13.

Call anytime (now through December 9) at 877-518-0606 and record your message.

Learn how you can help blog, distribute, broadcast, or support this event.

CALLS FROM HOME is a project of Thousand Kites/WMMT-FM/Appalshop and a national network of grassroots organizations working for criminal justice reform.

www.thousandkites.org
thousandkitesproject@gmail.com

In Peace,
Thousand Kites Team (Nick, Julia, Dudley, Donna, and Amelia)

Thousand Kites
91 Madison Ave
Whitesburg, KY 41858

Reading across the lines

The book group I’m facilitating at the county jail met again this past week, interrupting an Uno game going on in the common area of the cell block.  I had just played a game of Uno that afternoon, sprawled out across a post-picnic blanket on a grassy patch just above the lake.  Kids splashed below in the warm water, teenage girls lounged allofly in inflatable furniture, and in the distance, people careened back and forth across the wake of motorboats.  This is one of the things that is the most difficult about going into the jail – things that are completely familiar to me, like the Uno game, in a context that is completely different to my everyday reality.  Actually going into the jail has made me realize the boundary between solidarity and being in the same boat.  I think everything that I’ve encountered about jails and prisons firmly establish where you stand.  You are a guard, you are incarcerated, you are incarcerated and in the “therepeutic” block, you are in solitary, you are in the general population, you are a family member or friend here for visiting hours, you are a volunteer.  The roles, the mobility associated with each, and the expectations of each group by the others seem hard and rigid, ruts torn deeper and deeper with all the inertia of the prison industrial complex.  I want to just think that I’m just down, but even going in with the interest of supporting incarcerated folks is mediated by the fact, told bluntly by the corrections officer who did the volunteer training, that my ability to go in is largely based on the realization that programming in the jail tends to placate the incarcerated population there.

The reading group happens in the block and has a variable number of participants.  This is affected by what people have going on at the moment and the fact that people are constantly coming in and out of the jail, coming from a DOC facility for court or returning to a DOC facility after court.  This week, only 4 folks sat down at the table, with only 1 of them having read the book to be discussed.  The book might have been part of the problem.  A few weeks ago we had decided to read A Walk in the Woods by Bill Bryson, a travel account of two middle-aged men who decide to hike part of the Appalachian Trail.  It was funny enough, and met the criteria that was established by those initially interested in the group, of having a subject that seemed to be far, far from the reality of incarceration and didn’t have the self-help twelve-steppy overtones that mark some of the other programming they participate in.  Still, a few folks expressed that they couldn’t get into the book.  My favorite response to the book that I heard was one word – “quitters”, as the protagonists seemed to spend as many nights sleeping in hotels and eating at diners as they did in the woods.  As I read through the book myself, I was a little worried that the book centered on this activity that represented an idleness and mobility; to spend weeks just hiking without worrying about jobs, families, or the things that a lot of the things that people in the jail talk to me about as concerns; that it would seem just insensitive.  Still, it’s unfair and just not true to make assumptions about people’s situations or their reactions in the context of their lives.

So what is my place here in the jail?  Reading, for all the reasons that anyone loves it, with the additional weight of it being an activity that can happen, relatively unhindered, even within the constraints of incarceration, seems to be an important part of people’s lives at the jail.  People talk about the books that they read, and pass them around.  I remember a conversation starting with “Remember that Marilyn Manson biography …”, the book having apparenly made its rounds through the block.  There’s no question that books are important to people in jail.  Having someone come in to facilitate a discussion group about books seems of more questionable value. For the men in the cell block where the programming happens, their days are filled with different groups, many of them focussed around rehabilitation, I think that one more structured activity that involves a group and a discussion just doesn’t seem that appealing.  Volunteering in the jail, there seems to be such an impasse between what corrections officials and non-profits think people who are incacerated need and what people who are incarcerated say they need.  I think that more than anything, people need to not be incarcerated, because dealing with all the other things in life become frustratingly cumbersome to impossible.  Beyond that I think the concerns of incarcerated people are the same as a lot of people that I know, obviously with varying degrees of severity: economic security, a safe, comfortable place to live, help sorting out relationships and family.  Those are such large, ambiguous things, but it’s the way I can most accurately express it.

I like going into the jail because it has made me have to reevaluate how I think about other people and about prison issues as “issues”.  But, even though I feel like I’m getting something out of my volunteer work, the exchange doesn’t exactly seem equal.  One of the men interested in the reading group said he would talk to others in the block to guage the actual interest and to get some input about what format would be best.  We talked about two things that would be an improvement – meeting once a month instead of biweekly and reading shorter works of fiction.  One thing that seems like something that I can really offer is just giving people access to books.  The Monroe County Public Library has a sweet jail library program, but the men I work with said that while they used to get to go to the library once every two weeks, they now can only go once every 4 or 5 weeks.

I’ve also sporadically tutoring math in the jail, and had been working with someone who just passed his GED exams.  Working in this capacity seems like I can offer people something of myself that seems more useful, but it’s still hard.  The person who just passed his GED said that one of the reasons he wanted to get his GED was so he could go into the armed forces when he was released.  I don’t want to see anyone join the armed forces, but I’m afraid that, facing the realities of the current economy, and the additional challenges that someone with a criminal record faces getting a job, the options are limited.  It’s so frustrating that I, and the things I believe in, can’t offer an alternative.  This makes me feel like I’m not in a position to do what people really need.  I can give people books, or some tricks to solving math problems, but I can’t give people jobs or build houses.  It makes me feel like I’m not doing the right thing. Everyone, and I mean everyone everyone and not just incarceration people, need inspiration and tools, to be sure, but it seems really narrow sighted to think that they’re enough.

diners, corrections, and community

I’ve always loved diners, ever since I was a kid and going to Tucker’s this weekend on a trip to Cincinnati made me think about the role diners and other meeting places play as community anchors.  Then I saw the following article about a Rhode Island program for incarcerated youth that has them learning carpentry and other building skills to restore classic diner cars.  This seems cool, or at least useful, and the response from participants positive, but I always have mixed feelings about programming within the prison system, despite my investment in that.

Youth in RI juvenile detention centers work on restoring diners and preparing food.

Youthful Offenders Restoring Luster to Diners of Old – NYTimes.com:

“It keeps my mind off the negative,” said Rob (state officials withheld last names because juvenile offenders’ records are not public). “I can say, ‘Yeah I helped make that.’ ”

Rob, who said he had “been in detention a million times,” said he preferred the diner work over some other training programs, like the poetry and rapping workshop, which he said censored some language.

“You can’t express what you want — nothing about drugs, violence, sex,” said Rob, who plans to record a rap album and call it “In Harm’s Way, Volume 1: Talk Is Cheap.” “They just want us to rap positive. But I can’t just be talking about sunshine and flowers and how colorful they are. That’s not my life experience.”

Devin, 18, with a 10-year history of armed robbery and lesser crimes, said he also values the diner program, especially taking apart the stove and refrigerator because, “I always want to break things.”

“Building birdhouses like a traditional high school program is not what these kids need,” Mr. Scott said. “We’re actually preparing them for all kinds of skills: there’s ceramic tile in these diners, sheet metal work, plumbing, electrical. You always meet people who want these kids to be locked away, and I respect their ill-informed opinion. But I look at the training school as kind of like Home Depot of the correctional system. We give them the tools, and when they’re ready to use it, they’ll use it.”

To keep the cash-strapped state from paying $25,000 to $200,000 to restore each diner, Mr. Scott and Mr. Zilka found partners in Rhode Island. New Harvest Coffee Roasters in Pawtucket concocted New Hope coffee (organic, fair trade, shade grown), and about $4 of each $11 bag supports the project.

Students at Bryant University in Smithfield are creating business plans for each diner. Angelo’s, an 84-year-old landmark restaurant in Providence, will operate the diners, employing offenders once they are released.

“Hopefully, one of them will be able to own one,” said Robert Antignano, the president of Angelo’s.

Jail Book Group

I’m trying to be better about posting what I’ve been doing lately.  Last night, the book group I’m facilitating through Pages and New Leaf New Life in the “therapeutic” block of the local jail met for the second time and we picked the book that we’re going to read, A Walk in the Woods by Bill Bryson.  We did a rough vote and there wasn’t an overwhelming consensus so I’m going to bring in a few copies of the other books that I brought up as options including Me Talk Pretty One Day by David Sedaris, Oryx and Crake by Margaret Atwood, and The Golden Compass by Phillip Pullman.

Doing the group is challenging.  Some people are extroverted and seem to love to talk about themselves and their experiences.  Some are the exact opposite.  I feel like we’re also fighting the difficult dynamic of being a “group” amidst a lot of other mandatory groups that the men have to go through all day.  I think, at the end of the day, some just aren’t feeling another group.  People in the block are respectful and quiet, but the whole jail is noisy.  There are lots of interruptions like meds and the church group that comes in to provide worship services without notices.  I’m still getting my balance as a facilitator and trying to make it more clear why I’m there and what I’m doing and try to get past the reasonable distrust that some of the guys have for people like me.

In spite of all the challenges, we had a short discussion about a piece of writing titled The Best Time in My Life and many shared a memory or description of places and eras that they had seen pass.  For some it was rock quarries in southern Indiana, for another being towed around on an old car hood in his tiny hometown, and for another it was the closure of a vital youth center in his Chicago neighborhood.

A New Jail?!: Exploring Alternatives to Incarceration in Monroe County

May 3 Jail Education Banner
Saturday, May 3, 2008
Free
A New Jail?!: Exploring Alternatives to Incarceration in Monroe County

Events At:

Boxcar Books
310A S. Washington St.
Bloomington, IN 47401
(812) 339-8710

Home

The Cinemat
123 S. Walnut St.
(812) 333-4700
Bloomington, IN 47401
http://www.thecinemat.com

A day of popular education that will explore alternatives to the current proposal of building new adult and youth jails in Bloomington. Through interactive workshops featuring community members, non-profits, experienced activists, and YOU, we will explore: What is the history of incarceration in Monroe County? What are some existing community alternatives to incarceration in Bloomington and other parts of the country? What have successful campaigns to challenge other jail constructions looked like? Several workshops will offer education, training, and networking opportunities for local folks questioning the need for new jails. The day’s events will culminate with special guest speaker Kai Barrow from Critical Resistance. This is an open event, with people from all backgrounds and political beliefs encouraged to attend.

Workshop One- 11:00 am, Boxcar Books
A discussion and Presentation on the History of the Monroe County Jail System

This workshop will begin with an interview from Indiana University Criminal Justice Department faculty member, Hal Pepinsky. Following the interview will be a facilitated discussion on the history of the jail, encouraging input from community members in piecing together the jail’s history and the sharing of personal and familial experiences with the jail.

Workshop Two- 2:00 pm, Boxcar Books
A Panel Discussion on Alternatives to Incarceration

This workshop will explore community alternatives to incarceration in Bloomington, as well as highlight successful initiatives from around the country. Representatives from different organizations will speak briefly about how their work keeps people out of jail by offering positive responses to poverty, homelessness, hunger, and conflict that enrich and restore community. Workshop attendees will be able to offer their own perspectives on what makes a community safe and whole.

Film Screening- 4:00 pm, The Cinemat
Yes, In My Backyard, A documentary by Tracy Huling

Financially-strapped communities are now begging for prisons to be built in their backyards. With plundering agribusiness, military-base closings and major industry relocation, incarceration is seen as the salvation of rural communities. Through the eyes of one farming-community-turned-prison-town, this hour documentary explores the increasing and multi-layered dependence of rural America on prison industries and subtly probes the profound implications of this dependence for both the keepers and the kept, and for our society’s understanding of and response to crime.

Workshop Three- 6:00 pm, Boxcar Books
Potluck Dinner and Discussion with Kai Barrow from Critical Resistance

Kai Barrow, one of five paid staff persons for Critical Resistance, a national grassroots organization committed to ending society’s use of prisons and policing as an answer to social problems, will lead a discussion

For more press, interview, or further contact information, please use information below:

Judah Schept
(812) 219-3611
jschept@indiana.edu

Download a flyer for this event (PDF)

Mail a letter to Lambert Day

Lambert used to live in Bloomington, but, due to personal issues, movee away to join the Navy.  His mailing address is:

S R Daniel-Marks, Lambert A
Division 933
Ship 06
Recruit Training Command
3510 illinois St
great lakes, IL
60088

half.com quick sell mycroft plugin

So Pages to Prisoners sells books that seem somewhat valuable and probably won’t get requested by folks in prison on Half.com as a fundraiser.  All proceeds go into supplies and postage for the project.  We get a lot of books sometimes, and I wanted to streamline the price lookup process.  So, I made a MyCroft plugin for Half.com’s quick sell form.  You can install the extension here.

pages ISBN to title lookup info

Alex Berry’s Z39.50 to web gateway: http://www.arthurspilling.com/ISBN/zoomtst2.cgi

Lisa pointed me to everything I’d ever want to know about MARC records
www.oclc.org/bibformats/en

Some field codes:

  • 100 author
  • 245 title
  • 650 subj
  • 260 publisher
  • 020 ISBN