Professional Software Development for FileMaker®

FileMaker Business Alliance
FileMaker 8 Certified Developer
FileMaker 10 Certified Developer
SUI Solutions is an independent entity and this web site has not been authorized, sponsored, or otherwise approved by FileMaker, Inc.

New to FileMaker?

FileMaker 11

Learn about the most
incredible database now!

Tokenize

Filemaker custom function that returns value list of all tokenized words from Text.

Format

Tokenize ( Text ; Separator )

Parameters

Text - source text
Separator - condition are used as a word separator

Data type returned

text

Description

Splits the Text on single words/phrases using the Separator.
If the empty Separator is used ("") then space (" "), dot ("."), and colon (",") are used as a word separator simultaneously.
The words are returned as a value list with carriage returns.

Example

Input (source text with conditions separated with "*"):

Tokenize ( "john_smith*some text about FileMaker solutions*family’s friend*some value" , "*")

Output (list of tokenized conditions):

john_smith
some text about FileMaker solutions
family’s friend
some value

Source code

Select all
Let([ space = " "; spacedText =If( $Already_Sub="yes";Text; Substitute ( text ; "." ; space )); spacedText = If( $Already_Sub="yes";spacedText; Substitute ( spacedText ; "," ; space )); $Already_Sub = "yes"; flag = If (IsEmpty(separator);0;1); adjustedText = If(flag; text; spacedText ); count = PatternCount(adjustedText;If(flag; Separator; space)); first = Position(adjustedText; If(flag; Separator; space) ; 1 ; 1 ); word = Left( adjustedText ; If(first = 0 ;Length ( adjustedText ); first-1)); adjustedText = Right (adjustedText; Length ( adjustedText ) - first); $Result = If ( IsEmpty(word) ; $Result ; $Result&word&¶ ) ]; If(count ≠ 0; Tokenize(adjustedText; Separator);Left($Result; Length ($Result )-1)))