opensearch filters - procure-project/EU-Contract-Hub GitHub Wiki

OpenSearch Filter System Guide

Overview

The filter system in the EU Contract Hub enables you to refine procurement data searches by applying specific criteria to your queries. Filters help you narrow down large datasets to focus on contracts that match your exact requirements.

Unlike basic text searches, filters operate on structured data fields, allowing for precise data selection without affecting relevance scoring. This makes them ideal for procurement analysis where you need to identify contracts with specific attributes.

Filters can be combined to create complex queries, saved for future use, and shared with colleagues for collaborative procurement analysis.

Filter Types

Basic Filters

Value Filters

Value filters match exact values in specific fields, ideal for filtering contracts by known criteria.

  • Exact Match: Select contracts with a specific value in a field
    • Example: Filtering by a specific country or procurement procedure
  • One of Many: Select contracts where a field matches any value from a list
    • Example: Filtering by multiple CPV codes or several countries
  • Exists/Missing: Find contracts where a field either exists or is missing
    • Example: Identifying contracts with or without lot information

To find all French procurement contracts:
Field: Country
Operator: is
Value: France

Range Filters

Range filters select contracts where numeric or date values fall within specified boundaries.

  • Greater Than/Less Than: Select contracts above or below a threshold value
    • Example: Contracts with values over €1 million
  • Between: Select contracts with values falling between two points
    • Example: Contracts valued between €500,000 and €2 million
  • Statistical Ranges: Select values based on statistical measures
    • Example: Contracts with values in the top quartile

To find high-value contracts:
Field: Value
Operator: is greater than
Value: 1000000

Date Filters

Date filters help analyze procurement trends and timeframes.

  • Absolute Dates: Select contracts from specific dates or date ranges
    • Example: Contracts published in 2023
  • Relative Dates: Select contracts from relative time periods
    • Example: Contracts published in the last 6 months
  • Date Comparisons: Compare dates across different fields
    • Example: Contracts where publication date and award date are more than 30 days apart

To find contracts published in the first half of 2023:
Field: Dispatch Date
Operator: is between
Values: 2023-01-01 to 2023-06-30

Advanced Filters

Complex Conditions

Complex filters combine multiple conditions using logical operators.

  • AND Logic: Requires all conditions to be true
    • Example: Contracts from France AND with values over €1 million
  • OR Logic: Requires at least one condition to be true
    • Example: Contracts from France OR Germany
  • NOT Logic: Excludes matching records
    • Example: All contracts except those from France

To find high-value medical contracts excluding France:

  1. Field: CPV | Operator: begins with | Value: 33 (Medical equipment)
  2. Field: Value | Operator: is greater than | Value: 1000000
  3. Field: Country | Operator: is not | Value: France

Nested Filters

Nested filters target data within arrays of objects, such as contract lots.

  • Lot Filtering: Filter contracts based on properties of individual lots
    • Example: Contracts with at least one lot valued over €500,000
  • Contractor Filtering: Filter by properties of participating companies
    • Example: Contracts with contractors from a specific country
  • Document Filtering: Filter by properties of attached documents
    • Example: Contracts with PDF attachments

To find contracts with high-value pharmaceutical lots:
Field: Lots.Value
Operator: is greater than
Value: 500000
AND
Field: Lots.CPV Codes
Operator: begins with
Value: 33600 (Pharmaceutical products)

Custom Filters

Custom filters allow for specialized search criteria beyond standard options.

  • Script Filters: Advanced filtering using custom logic
    • Example: Contracts where the award value is significantly higher than the estimate
  • Geospatial Filters: Filter based on geographic locations
    • Example: Contracts within 50km of Brussels
  • Term Proximity: Filter by term proximity in text fields
    • Example: Contracts where "medical" and "equipment" appear within 3 words of each other

To find contracts from authorities near Brussels:
Field: Contracting Authority.Address.Territorial Unit (NUTS3)
Operator: is
Value: BE100 (Brussels region)

Filter Management

Filter Creation

Creating effective filters follows a consistent process:

  • Field Selection: Choose the appropriate field to filter on
    • Core contract fields (e.g., Title, Value, Dispatch Date)
    • Category fields (e.g., CPV, Contract Nature)
    • Authority fields (e.g., Contracting Authority.Name, Contracting Authority.Address.Country)
    • Lot fields (e.g., Lots.Title, Lots.Value)
  • Operator Selection: Choose the relationship between field and value
    • Equality operators (is, is not, is one of)
    • Comparison operators (greater than, less than, between)
    • Text operators (contains, does not contain, matches pattern)
  • Value Input: Specify the filtering criteria
    • Single values for exact matches
    • Multiple values for "one of" conditions
    • Range boundaries for between conditions

Filter Storage

Filters can be saved for later use:

  • Dashboard Filters: Saved with dashboards for quick access
    • Ideal for standard procurement analysis workflows
  • Global Filters: Available across all dashboards
    • Useful for organization-wide procurement criteria
  • User Filters: Personal filters saved to your profile
    • For individual procurement analysis needs

Filter Sharing

Filters can be shared with colleagues:

  • URL Sharing: Generate URLs with embedded filter settings
    • Recipients see the exact same filtered view
  • Dashboard Sharing: Share dashboards with predefined filters
    • Recipients can interact with and modify the filters
  • Export Options: Download filtered data in various formats
    • CSV, Excel, or PDF formats for further analysis

How to Use Filters

1. Basic Filtering

  1. Filter Selection

    • Click the "Add filter" button in the navigation bar
    • Select a field from the dropdown menu
    • Choose from frequently used fields or search for specific fields
    • Fields are organized by categories (Contract, Authority, Lots, etc.)

  2. Value Setting

    • Select an operator appropriate for the field type
    • Enter values matching your search criteria
    • For text fields, consider case sensitivity options
    • For date fields, use the calendar picker or relative date options

  3. Application Steps

    • Click "Apply" to activate the filter
    • The dashboard immediately updates to show only matching contracts
    • The active filter appears in the filter bar at the top of the page
    • Click the filter pill to edit or remove it

Example: Filtering for French healthcare contracts

  1. Click "Add filter"
  2. Select "Country" field
  3. Choose "is" operator
  4. Enter "France" as value
  5. Click "Apply"
  6. Click "Add filter" again
  7. Select "Healthcare CPV" field
  8. Choose "is" operator
  9. Select "true" value
  10. Click "Apply"

2. Advanced Filtering

  1. Complex Filter Creation

    • Use the "Add filter" process for each condition
    • Set the relationship between filters (AND/OR)
    • Group related filters together for logical organization
    • Use parentheses to establish precedence in complex conditions

  2. Combination Methods

    • Filter Groups: Create logical groups of related filters
      • Click the gear icon in the filter bar
      • Select "Group filters" option
      • Select filters to include in the group
    • Filter Logic: Change the logic between filters
      • Default relationship is AND (all filters must match)
      • Change to OR for more inclusive results
      • Mix AND/OR relationships for complex logic

  3. Filter Optimization

    • Apply the most restrictive filters first to improve performance
    • Use exact match filters rather than text search where possible
    • Consider the impact of wildcard and fuzzy searches on performance
    • Limit the number of OR conditions for better responsiveness

Example: Finding high-value medical contracts from specific countries

  1. Create a filter group for countries:
    • Field: "Country" | Operator: "is one of" | Values: "France,Germany,Italy"
  2. Create a filter for contract value:
    • Field: "Value" | Operator: "is greater than" | Value: "1000000"
  3. Create a filter for medical equipment:
    • Field: "CPV" | Operator: "begins with" | Value: "33"
  4. Group all filters with AND logic

Example Scenarios

Scenario 1: Simple Filtering

Objective: Find all healthcare procurement contracts from Spain published in 2023.

Filter Workflow:

  1. Add Country filter:
    • Field: "Country"
    • Operator: "is"
    • Value: "Spain"
  2. Add Date filter:
    • Field: "Dispatch Date"
    • Operator: "is between"
    • Values: "2023-01-01" to "2023-12-31"
  3. Add Healthcare filter:
    • Field: "Healthcare CPV"
    • Operator: "is"
    • Value: "true"

Results Review:

  • The dashboard updates to show only Spanish healthcare contracts from 2023
  • Visualizations recalculate to reflect the filtered dataset
  • Contract count and value totals update to reflect the filtered subset
  • You can now analyze trends, procurement types, and value distributions within this specific segment

Scenario 2: Complex Filtering

Objective: Analyze high-value medical equipment contracts with multiple lots from three specific EU countries.

Filter Setup:

  1. Create Country filter group:
    • Field: "Country"
    • Operator: "is one of"
    • Values: "Germany,France,Italy"
  2. Add CPV filter:
    • Field: "CPV"
    • Operator: "begins with"
    • Value: "33" (Medical equipment code)
  3. Add Value filter:
    • Field: "Value"
    • Operator: "is greater than"
    • Value: "1000000"
  4. Add Lots filter:
    • Field: "LotsNumber"
    • Operator: "is greater than"
    • Value: "1"

Output Analysis:

  • Compare contract values across the three countries
  • Analyze the distribution of medical equipment subcategories
  • Identify patterns in lot structuring for high-value contracts
  • Export filtered data for detailed offline analysis

For guidance on specific filter applications or advanced query techniques, consult the Dashboard Query Language Guide or Data Fields Reference.