The gdbm Module

The gdbm extension command is provided by the external Gdbm Tcl module. It must be loaded before the command become available.

Example for Tcl :

package require Gdbm

and for Python

from gdbm import *

Gdbm files need to be opened or created before any operations on them can commence. Afterwards, the access object is identified by a handle, which is returned by the opener commands. When they are no longer in use, they should be closed. Gdbm files have different internal structure on 32 vs. 64 bit systems, and are byte-order-dependent. They are therefore best suited for local, temporary files.

The general syntax of the gdbm command follows the usual command/subcommand/handle/parameters syntax. The gdbm command is to a large degree compatible to the newer and generally preferred tc command, which performs the same types of operations on Tokyo Cabinet files. The latter command can be significantly faster for large files, generated more compact data files. Additionally, these data files are 32/64-bit clean and byte-order-independent and thus much more portable between systems.

Note that the Python version of this module is not identical to the standard Python gdbm module.

Examples:

set gdhandle [gdbm open mygdbmfile.gdb]
set data [gdbm get $gdhandle $key]
gdbm close $gdhandle

This is the list of subcommands:

gdbm add

gdbm add gdhandle ?-nocase? key ?data?...
g.add(key=,data=,?nocase=?)

Append the listed data items as Tcl list elements to the entry identified by the key. If no such record exists, a new record with the initial set of list items is created if there are data arguments. If any of the data items are already present in the current value list, they are ignored, so duplicates are not added. The duplicate check is performed in case-insensitive fashion if the - nocase flag is used.

The gdbm append command performs a similar operation, but without a duplicate check.

The command returns the updated entry value list.

This command only works if an existing value is a properly formatted Tcl list. For Python , you can either pass a list or tuple, or a string as data argument which will be split into list items according to the Tcl syntax.

Example:

gdbm add $gdhandle “project_$projectid” [ens get $eh E_IDENT]
gdbm add $gdhandle “project_$projectid” [ens get $eh E_IDENT]
 

The second statement has no effect.

gdbm append

gdbm append gdhandle key ?data?...
g.append(key=,data=)

Append the data items as Tcl list elements to the entry identified by the key. If no such record exists, a new record with the initial set of list items is created if there are any data arguments. No duplicate check is performed for the added list elements - this is the difference to the gdbm add command.

The command returns the updated entry value list.

This command only works if an existing value is a properly formatted Tcl list. For Python , you can either pass a list or tuple as data argument, or a string which will be split into list items according to the Tcl syntax.

Example:

gdbm append $gdhandle “project_$projectid” [ens get $eh E_IDENT]
gdbm append $gdhandle “project_$projectid” [ens get $eh E_IDENT]

The second statement adds a duplicate list element to the entry.

gdbm close

gdbm close ?gdhandle?...
gdbm close all
g.close()
Gdbm.Close(?gref/ghandle?,...)
Gdbm.Close(“all”)

This command closes opened Gdbm files. After a file has been closed, its handle becomes invalid, but may the reissued for another opened Gdbm file again.

The first version of the command closes specific files. The second version closes all open Gdbm files in the current application. Both variants return the number of closed files as result.

Example:

gdbm close $gdhandle

gdbm count

gdbm count gdhandle ?pattern?
g.count(?pattern=?)

Count the number of keys in the file. If a pattern argument is given, only the file entries with a matching key are counted. The command returns the number of passing keys.

Unfortunately, Gdbm files do not maintain an internal record count, so this command has to loop through all keys, which can take a substantial amount of time for large files.

The command gdbm size is an alias for this command.

Example:

set size [gdbm count $gdhandle]

gdbm create

gdbm create filename ?mode? ?filemode? ?blocksize?
Gdbm.Create(filename=,?mode=?,?filemode=?,?blocksize=?)
Gdbm(filename=,?mode=?,?filemode=?,?blocksize=?)

This command is the same as the command gdbm open , except that the default file access mode to be used if no explicit mode is specified is create instead of read .

Example:

set gdhandle [gdbm create thefile.gdb]

gdbm delete

gdbm delete gdhandle key ?ispattern?
g.delete(key=,?ispattern=?)

If no ispattern boolean argument is given, or it is not a true value, the record matching the key exactly is deleted. If the entry did exist and could be deleted, boolean 1 is returned, 0 otherwise.

If the ispattern flag is set, the key argument is interpreted as a string match pattern. All records with keys matching the pattern are deleted. In this case, the return value is the number of deleted entries.

Example:

gdbm delete $gdhandle count* 1

deletes all records with keys starting with count .

gdbm dump

gdbm dump gdhandle ?file/pipe/std_channel/tcl_channel? ?keypattern? ?datapattern?
g.dump(?filename=?,?keypattern=?,?datapattern=?)

If no file or channel handle is specified, or an empty string is used (or None for Python ), this command is used to obtain the complete contents of a Gdbm file as a dictionary. If non-empty filter patterns are set, only those key/value pairs where the key or data part matches the respective pattern are reported.

If a file name, a Unix pipe in standard notation, a standard channel or a valid Tcl channel handle argument is specified, and the target is writable, the filtered key/value pairs are written to that channel. Every key/value pair is formatted as a simple two-element Tcl list (even in the Python interface) and written to the file. In this mode, the return value is the number of lines written.

Example:

gdbm dump $gdhandle stdout

gdbm exists

gdbm exists gdhandle key
g.exists(key)

This command returns boolean 1 if the key is in the Gdbm file, 0 otherwise.

Example:

set isknown [gdbm exists $gdhandle [ens get $ehandle E_HASHY]]

gdbm first

gdbm first gdhandle ?pattern?
g.first(?pattern=?)

Returns the key of the first file entry. In case a pattern is specified, the first entry whose key matches the pattern is retrieved. If no matching or first key can be found, an error results.

Example:

gdbm first $gdhandle Z*

returns the first key which starts with a (case-sensitive) Z. Keys are returned in an unpredictable order.

This command is typically used in combination with subsequent gdbm next commands.

gdbm get

gdbm get gdhandle key ?silent? ?defaultvalue?
g.get(key=,?silent=?,?default=?)
g.key
g[key]

Retrieve the data value associated with the specified key. If the key does not exist, and a default value was specified, the default value is returned instead. Without a default value, the result is an empty string ( Tcl ) or None ( Python ) if the silent flag is set, and an error otherwise.

This command may return binary data with non-printable characters, so the language objects are byte arrays, not strings.

For Python , if the key is the same as the name of one of the class methods, only g.get() can be used to retrieve the data.

Example:

gdbm get $gdhandle [gdbm first $gdhandle]

gdbm incr

gdbm incr gdhandle key ?delta?
g.incr(key=,?delta=?)

Increment the data value stored under the specified key by the delta value. If no delta value is specified, it defaults to one. If the key does not exist, a new key/value pair with an initial data value of the delta is created. If the entry exists, but the data value is not a valid integer, an error is raised. The value is stored as a string.

The return value of this command is the incremented value.

Example:

gdbm insert $gdhandle “count_dracula” 5
puts [gdbm inc $gdhandle “count_dracula”]

This command sequence updates the file, and outputs 6.

gdbm index

gdbm index gdhandle index ?pattern?
g.index(index=,?pattern=?)

Return the nth key in the database if no pattern is specified, or the nth key which matches the pattern. The key index starts with zero.

Example:

gdbm index $gdhandle 10 cpdname*

returns the key for the eleventh file record which starts with the string cpdname. If no such key exists, an error is raised.

gdbm insert

gdbm insert gdhandle ?key value?...
gdbm insert gdhandle dict
g.insert(?key,value?,...)
g.insert(dict)

Insert one or more new key/value pairs into the file. The values may be binary data. If a key already exists, the return value for that key/value pair is boolean 0 and the old data is not overwritten. If a key is not in the database, boolean 1 is returned for that item pair and the data is stored in the file.

If only a single argument is used after the handle, it is expected to be a properly formed dictionary. In that case, all dictionary elements are processed as if they were spelled out as individual key/value pairs.

The return value is a list of boolean flags indicating success or failure for the operation on each key/value pair. No error is raised if a write operation fails.

Example:

gdbm insert $gdhandle [ens get $ehandle E_HASH] [ens get $ehandle E_GIF]
gdbm insert $gdhandle [array get ::params]

gdbm keys

gdbm keys gdhandle ?pattern?...
g.keys(?pattern=?)

This command returns a list of all keys in the file. Keys may optionally be filtered by one or more string match patterns. The Python version only supports a single pattern.

The related gdbm match command can be used to obtain a list of keys which are filtered by the value component and not the key name.

Example:

set keylist [gdbm keys $gdhandle project${id}*]

gdbm linkvar

gdbm linkvar gdhandle ?-loadall? ?-preserve? varname

This command establishes a link between a Gdbm file and a Tcl array variable. If the variable is already in existence, it is deleted prior to recreation as a linked array variable. This can be prevented by using the - preserve option. This command is not supported with Python .

After the link has been formed, any read access by a Tcl script to an element of the array variable retrieves the data value from the file which corresponds to the array element name as key, if the variable element is not yet set. If the variable element already exists, it takes precedence over the file contents, but this can only happen if the - preserve option was used when the variable was linked. In case file retrieval is initiated, and the key does not exist in the file, an error is generated.

Assigning a value to an array element replaces or creates the corresponding file entry. If an array element is deleted (for example by the Tcl unset command), the corresponding key and value in the file are also deleted. Replacement and deletion operations require a file opened for write access.

If the -loadall option is used, all array elements are immediately loaded by looping over all file keys when the command is run. By default, data is retrieved from the file only when an array element is explicitly accessed. If this option is used, the array names command immediately shows all file keys, not just the ones currently loaded.

Since Gdbm data, once retrieved from the file by accessing the linked array element, is never deleted from memory, the use of this utility is not recommended for large files.

It is possible to link multiple variables to a single Gdbm file.

Gdbm files must not be closed when variable links to the closed file are active. Currently, there is no mechanism to detect that the target of a variable link as gone away, so this cannot be handled automatically. Variable links can be removed by the gdbm unlinkvar command.

Example:

gdbm linkvar $gdhandle g_array
puts “Key: $thekey Value: $g_array($thekey)”
set g_array($newkey) $newvalue
unset g_array($thekey)

gdbm list

gdbm list gdhandle ?pattern?
Gdbm.List(?pattern=?)

Get a list of currently active Gdbm file handles. If no files have been opened, an empty list is returned. Optionally, a string filter pattern can be specified.

Example:

gdbm list

gdbm loop

gdbm loop gdhandle keyvar datavar ?pattern? body
g.loop(function=,?keyvariable=?,?datavariable=?,?pattern=?)

Run a loop over all file entries. If a string filter pattern is specified, only those records with a matching key are visited. After retrieving the key and data values, the key and data variables are initialized to the current key and data values, and then the commands in the body section are executed.

With Tcl , in the body, the standard break or return statements may be used to force an early exit from the loop, and the continue statement also works as expected. In case an uncaught error occurs in the body, the loop terminates with an error message.

The Python interface intentionally has a different argument sequence. The function may either be a function reference, or a multi-line string containing the loop body code similar to the Tcl style. If a function reference is used, the function is called with a key/value tuple as only argument. Loop variables are not required in this mode (though they are still supported). Normal break and continue loop control statements cannot be used in the Python functions. Instead, the custom BreakLoop and ContinueLoop exceptions can be thrown for the same effect.

If the loop was not terminated due to an error, the result value is the number of visited keys.

Example:

gdbm loop $gdhandle key data {
	puts “Key: $key with value $data”
}

gdbm match

gdbm match gdhandle ?pattern?
g.match(?pattern=?)

This command returns a list of all keys where the data value matches the filter pattern. This is similar to the gdbm keys command, with the difference that in that command the key names are used for filtering, not the data values.

Example:

set keylist [gdbm match $gdhandle *nitro*]

gdbm new

gdbm new filename ?mode? ?filemode? ?blocksize?
Gdbm.New(filename=,?mode=?,?filemode=?,?blocksize=?)

This command is the same as the command gdbm open , except that the default file access mode to be used if no explicit mode is specified is new instead of read.

Example:

set gdhandle [gdbm new thefile.gdb]

gdbm next

gdbm next gdhandle key ?pattern?
g.next(key=,?pattern=?)

Get the next key after the specified key which matches the pattern. If no pattern is specified, no key filtering is performed. If no key can be found, an error is raised, otherwise the key is returned as function result.

A starting point for a key traversal is usually obtained via a gdbm first or gdbm index command.

Example:

gdbm next $gdhandle name20 name*

produces the next key after the current key name20 matching the pattern name*. This key is not necessarily name21 , or any other predictable value. Rather, the key sequence in the database is determined by its internal hash, and the ordering is pseudo-random. The only guarantee is that with a gdbm first/gdbm next loop all keys are ultimately visited. A deletion of the current key is allowed and guaranteed to not disturb the traversal sequence.

gdbm open

gdbm open filename ?mode? ?filemode? ?blocksize?
Gdbm(filename=,?mode=?,?filemode=?,?blocksize=?)
Gdbm.Open(filename=,?mode=?,?filemode=?,?blocksize=?)

Open an existing Gdbm file, or create a new one. The command returns a new object handle or reference. The default mode is read , opening the file for reading. In this case, the file must already exist, and have been written on a computer with the same byte ordering and integer size. The mode parameter may consist of a bitset of the following options:

It is sufficient to specify the first letter of the mode options. Combining any of the first four file access modes does not make sense, though. In standard applications, the mode set is either a single word describing the access mode, or a combination of the fast attribute and the access mode.

The optional filemode parameter determines the Unix-style octal file access bits which have an effect only in case a new disk file is created. Finally, the rarely used blocksize parameter determines the page block size of the Gdbm file and is again only used in case a new file is created. By default the value is 0, meaning that a reasonable default value depending on the type of file system the file resides on is chosen.

This command returns a Gdbm file handle in the form gdbm%d which can be used in subsequent gdbm commands, until the file is closed and the handle becomes invalid.

gdbm ref

Gdbm.Ref(identifier)

Python -only method to get a reference of the Gdbm object from its handle.

gdbm reorganize

gdbm reorganize gdhandle
g.reorganize()

Compact the data file. This can be useful after many deletions. The underlying Gdbm library copies the entries to another file, which ultimately replaces the original file. Thus, in worst case, sufficient disk space for two parallel files with the original file size must be available. This command requires write access to the Gdbm file.

gdbm reorg is an alias.

For large files, this command can take a long time to complete.

Example:

gdbm reorg $gdhandle

gdbm replace

gdbm replace gdhandle ?key value?...
gdbm replace gdhandle dict
g.replace(?key,value?,...)
g.replace(dict)
g.key = value
g[key] = value

Store one or more specified values which may be binary data, under the given keys. If a key is already in use, the old value is overwritten. If only a single argument is used after the handle, it is expected to be a properly formed dictionary. In that case, all dictionary elements are processed as if they were written out as individual key/value pairs.

The return value is a list of boolean flags indicating success or failure for the operation on each key/value pair. No error is raised if a write failed.

gdbm set is an alias for this command.

Examples:

gdbm replace $gdhandle $thekey $newdata $key2 $moredata
gdbm replace $gdhandle [array get ::params]

gdbm restore

gdbm restore gdhandle filename ?callback?
g.restore(filename=,?function=?)

This command reads a text file which usually was produced by the gdbm dump command and adds its contents to the current Gdbm file. Every line of the file is expected to contain a properly formatted two-element Tcl list. The list is split into the key and data parts and the contents are written to the Gdbm file, replacing old entries in case of existing keys.

The input file may be gzip-compressed or plain ASCII text.

If a callback function is specified, it is called with the Gdbm handle (or reference), the input file name, and the current restored item count after each successful line input.

The command gdbm readfile is an alias for this command.

The return value is the number of file lines read.

Example:

gdbm restore $gdhandle “we_will_never_need_this_backup.txt”

gdbm set

This is an alias for gdbm replace .

gdbm sync

gdbm sync gdhandle
g.sync()

Synchronize the in-memory and disk status of the file. All pending changes are committed to disk. This command has an effect only if the file was opened in fast mode (see gdbm open command).

gdbm synchronize is an alias.

Example:

gdbm sync $gdhandle

gdbm unlinkvar

gdbm unlinkvar gdhandle ?-preserve? varname

Unlink a Tcl array variable from a Gdbm file. If the - preserve option is not used, the variable is also deleted from the Tcl interpreter. The association between array variable and Gdbm file is also automatically broken when the variable is deleted by other means, such as a Tcl unset command.

The Gdbm file contents are not modified by unlinking by means of executing this command, or by any other methods of variable deletion.

Variable links are created with the gdbm linkvar command.

This command is not supported in Python .

Example:

gdbm unlinkvar $gdhandle g_array