- CLARIN-FCS Core 2.0 query language grammar and parser
- based on Github: clarin-eric/fcs-ql and Github: clarin-eric/fcs-simple-endpoint
- for more details visit: CLARIN FCS Technical Details
Install from PyPI:
python3 -m pip install fcs-ql-parserOr install from source:
git clone https://github.com/Querela/fcs-ql-python.git
cd fcs-ql-python
uv build
# built package
python3 -m pip install dist/fcs_ql_parser-<version>-py3-none-any.whl
# or
python3 -m pip install dist/fcs_ql_parser-<version>.tar.gz
# for local development
python3 -m pip install -e .The high-level interface fcsql.parser.QueryParser wraps the ANTLR4 parse tree into a simplified query node tree that is easier to work with. The fcsql-parser exposes a simple parsing function with fcsql.parse(input: str, enableSourceLocations: bool = True) -> fcsql.parser.QueryNode:
import fcsql
## parsing a valid query into a query node tree
# our query input string
input = '[ pos = "NOUN" ]'
# parse into QueryNode tree
sc = fcsql.parse(input)
# print stringified tree
print(str(sc))
## handling possibly invalid queries
input = "[ kaputt ]"
try:
fcsql.parse(input)
except fcsql.QueryParserException as ex:
print(f"Error: {ex}")You can also use the more low-level ANTLR4 framework to parse the query string. A handy wrapper is provided with fcsql.antlr_parse(input: str) -> LexParser.QueryContext.
from antlr4 import CommonTokenStream, InputStream
from fcsql.parser import FCSLexer, FCSParser
input = '"test"'
input_stream = InputStream(input)
lexer = FCSLexer(input_stream)
stream = CommonTokenStream(lexer)
parser = FCSParser(stream)
tree: FCSParser.QueryContext = parser.query()Parsed queries can also be checked against their specification conformance.
from fcsql import QueryParser
from fcsql.validation import FCSQLValidator, SpecificationValidationError
parser = QueryParser(enableSourceLocations=True)
query = '"Banane"'
node = parser.parse(query)
validator = FCSQLValidator()
validator.validate(node, query=query)
len(validator.errors) == 0 # no errors
# or to raise an error on first violation
query = '[ post = "NOUN" ]'
node = parser.parse(query)
validator = FCSQLValidator(raise_at_first_violation=True)
validator.validate(node, query=query) # raises SpecificationValidationErrorA convenience method is provded with fcsql.validate(query: str):
from fcsql import validate
# simple boolean returns
validate("'apples'") # => True
validate("apples") # => False (parse error, invalid construct, not a simple string or token)
validate('[ pos = "NOUNT" ]{3,0}') # => False (repetition max must be >= min)
# or with list of errors
error = validate("pos = NOUN", return_errors=True)[0] # has one error
error.message # "mismatched input 'pos' expecting {'(', '[', REGEXP}"
error.type # "syntax-error"
error.fragment # "pos"
error.position.start # 0 (start offset in query string)
error.position.stop # 3 ( end offset in query string)Fetch (or update) grammar files:
git clone https://github.com/clarin-eric/fcs-ql.git
cp fcs-ql/src/main/antlr4/eu/clarin/sru/fcs/qlparser/fcs/*.g4 src/fcsql/(Re-)Generate python parser code:
# setup environment
uv sync --extra antlr
# NOTE: you can activate the environment (if you do not want to prefix everything with `uv run`)
# NOTE: `uv` does not play nicely with `pyenv` - if you use `pyenv`, sourcing does NOT work!
source .venv/bin/activate
cd src/fcsql
uv run antlr4 -Dlanguage=Python3 *.g4 -listener -visitorRun style checks:
# setup environment
uv sync --extra style
uv run isort --check --diff .
uv run black --check .
uv run flake8 . --show-source --statistics
uv run mypy srcRun tests (pytest with coverage, clarity and randomly plugins):
# setup environment
uv sync --extra test
uv run pytest
# to see output and run a specific test file
uv run pytest -v -rP tests/validation/test_validation.py
# with logs
uv run pytest -v -rP -o log_cli=true -o log_cli_level="DEBUG"Build documentation:
# setup environment
uv sync --extra docs
# or if standalone
python3 -m pip install -r ./docs/requirements.txt
# build documentation and check links ...
uv run sphinx-build -b html docs dist/docs
uv run sphinx-build -b linkcheck docs dist/docsRun check before publishing:
# setup environment
uv sync --extra build
# build the package
uv build
# run metadata check
# uv run python3 -m build
uv run twine check --strict dist/*
# (manual) check of package contents
tar tvf dist/fcs_ql_parser-*.tar.gz