Full Text Searching with the Google App Engine Search API. Using with Flask to demonstrate.

example.txt

When going to /search/artist/<query_string> using the documents below you get:

	/search/artist/m	- this returns both Macklemore and Against Me!
	/search/artist/a	- this returns both Against Me! and AWOLNATION
	/search/artist/ma	- this returns only Macklemore
	/search/artist/aw	- this returns only AWOLNATION
	/search/artist/me	- this returns only Against Me!


Adding 'AWOLNATION' to models.Artist produces:

	index_spec=[name='api-artist]
	doc_id=['3861f55c-289f-5fb4-b738-a9c54d341da1']
	fields=[
		TextField[name='artist', value='AWOLNATION']
		TextField[name='pieces', value='A,AW,AWO,AWOL,AWOLN,AWOLNA,AWOLNAT,AWOLNATI,AWOLNATIO,AWOLNATION']
		TextField[name='id', value='3861f55c-289f-5fb4-b738-a9c54d341da1']
	]


Adding 'Macklemore' to models.Artist produces:

	index_spec=[name='api-artist]
	doc_id=['59a83c26-851a-5a21-8d2c-7107ca515994']
	fields=[
		TextField[name='artist', value='Macklemore']
		TextField[name='pieces', value='M,Ma,Mac,Mack,Mackl,Mackle,Macklem,Macklemo,Macklemor,Macklemore']
		TextField[name='id', value='59a83c26-851a-5a21-8d2c-7107ca515994']
	]


Adding 'Against Me!' to models.Artist produces:

	index_spec=[name='api-artist]
	doc_id=['183b7dba-976d-564b-bdb4-5865d90b34a3']
	fields=[
		TextField[name='artist', value='Against Me!']
		TextField[name='pieces', value='A,Ag,Aga,Agai,Again,Agains,Against,M,Me,Me!']
		TextField[name='id', value='183b7dba-976d-564b-bdb4-5865d90b34a3']
	]

models.py

from google.appengine.ext import ndb
from google.appengine.api import search

def pieces(string):
	pieces = []

	for word in string.split():
		cursor = 1
		while True:
			# this method produces pieces of 'TEXT' as 'T,TE,TEX,TEXT'
			pieces.append(word[:cursor])
			
			# optionally, you can do the following instead to procude
			# 'TEXT' as 'T,E,X,T,TE,EX,XT,TEX,EXT,TEXT'
			#
			# for i in range(len(word) - cursor + 1):
			#	pieces.append(word[i:i + cursor])
			
			if cursor == len(word): break
			cursor += 1

	return ','.join(pieces)

class Artist(ndb.Model):
	name			= ndb.StringProperty(required=True)
	name_lower		= ndb.ComputedProperty(lambda self: self.name.lower())
	# ...other keys...

	def _post_put_hook(self, future):
		doc = search.Document(doc_id=self.key.id(), fields=[
			search.TextField(name='artist', value=self.name),
			search.TextField(name='pieces', value=pieces(self.name)),
			search.TextField(name='id', value=self.key.id())])

		search.Index('api-artist').put(doc)

	@classmethod
	def _post_delete_hook(cls, key, future):
		search.Index('api-artist').delete(key.id())

views.py

from flask import jsonify
from google.appengine.api import search

from application import app, cache

@cache.memoize(timeout=60*60)
@app.route('/search/artist/<query_string>')
def search_artist(query_string):	
	sort = search.SortOptions(expressions=[
		search.SortExpression(expression='artist',
			direction=search.SortExpression.ASCENDING, default_value='')
		], limit=1000)

	options = search.QueryOptions(
		limit=10,
		sort_options=sort,
		returned_fields=['artist', 'id'])

	query_string = ''.join(['pieces:', query_string])
	query = search.Query(query_string=query_string, options=options)

	results = search.Index('api-artist').search(query)

	out = {'results': []}
	if results:
		for item in results:
			out['results'].append({f.name: f.value for f in item.fields})

	return jsonify(out)

Great example. But, for posterity, isn't line 26 of views.py supposed to be "for item in results.results"?

that's a very good example indeed.

Thanks for this example. Searched a lot but didn't find anything like this. Thanks :)

Thank you so much for this. The documentation of GAE is bad. Without this I couldn't have figured it out.

Thanks a lot. This post has been very very helpful.