EQL Roadmap - fieldenms/tg GitHub Wiki

I. Elimination of dependencies on EQL2 metadata and its removal.

- [ ] 1. Migrate IRetrievalModel to EQL3 metadata. - Separate issues/sub-tasks can be created for each of the seven fetch categories + one for populateProxies(). - [ ] 2. Migrate EntityContainerEnhancer to EQL3 metadata. - [ ] 3. Migrate DbCreator(s) to EQL3 metadata. - [ ] 4. Removal of EQL2 metadata and the related adjustments.

EQL2, EQL3 metadata were removed in favor of typeful metadata.

II. Other refactorings and adjustments that have been postponed to past convergence stage.

  • 5. Entity Query Progressive Interfaces (aka fluent interfaces) + Stage 0 to 1 transformation creation patterns understanding/refactoring.
    • This is critical for efficient (shorter and more accurate time estimates) enhancing of EQL with new operations (adding TOP, HAVING, CTEs, window functions, usual functions) and fixing the existing defect (caseWhen with complex conditions).
    • https://github.com/fieldenms/tg/issues/2173
  • 6. Adjust auto-yield to skip some types of implicitly calculated properties (composite key, union property common sub-properties) as in EQL2.
  • 7. In case of explicit yields remove those not contained within fetch model (as in EQL2).
  • 8. Reconcile logic within AbstractTokensBuilder.convertValue(..)/preprocessValue(..) with ValuePreprocessor.apply(..).
  • 9. Fix caseWhen to work with multiple conditions without conditions group.
  • 10. Substantially simplify internal logic once auto-addition of single member of CompositeUserType with single member (i.e., SimpleMoneyType .amount) is prohibited (it became obsolete after introduction of similar auto-addition for yielded money aliases.
  • 11. Adjust context processing logic for source-less queries to support extProp(..) within such queries (e.g., select().extProp("someProp")).
  • 12. Reuse instances of Expression2 and Expression3 that are generated for multiple occurrences of the same calculated property.
    • will result in same SQL expressions for multiple occurrences of the same calculated property (currently table and query aliases in such expressions are different due to unique numeration).
  • 13. Remove not used yielded non-calculated properties from source queries.
    • enables better performance and less cluttered SQL.

III. New end-developer features.

  • 15. Support for TOP with ORDER BY in sub-queries and source queries.
  • 16. Support for HAVING.
    • enables more expressive and faster queries
  • 17. Support for bushy JOINS.
    • allows flexibility while joining when needed for better performance
    • enables unit equi-testing for all kinds of EQL queries (see Equi-testing)
  • 18. Support for LATERAL JOINS.
    • enables more expressive and faster queries
  • 19. Support explicitly calculated collectional properties with ordering possibility (from Oles).
  • 20. Introduce a mandatory @Calculated annotation for marking implicit one-2-one calculated properties (implicit expression can be overriden by explicit one; the same is true for current collectional props that in fact are just implicitly calculated props).
  • 21. Ability to treat yielded entity ID as Entity Type in modelAsEntityAggregates() (i.e. be able to access entity subprops when such EA-query becomes source within another query).
  • 22. Support join(models...) similar to the existing select(models ..).
  • 23. Allow Lists in addition to Arrays in the EQL Fluent API.
  • 24. Support calculated props of Synthetic Entity type (those that have ID).
  • 25. Support proper type preservation for val(..) operator.
    • in ServiceAvailabilityUtils.mkQueryToSelectOutagesAndReleases val(true/false) is yielded via modelAsAggregate() and becomes "Y/"N" instead.
  • 26. Support convenient operators for algorithmic EQL generation (e.g., andConditions(List conditions) (from Vlad).
    • avoid contamination of client code with EQL Progressive Interfaces implementation details
    • ideally there should be no need to interrupt query composition with branching logic
  • 27. Passing context (YIELD/WHERE/etc) to such operators as prop()/val()/etc to enable other useful features (e.g., 25, 21).

IV. Improve test coverage.

  • 28. Cases of calculated properties in combination with dot-notation that need support for bushy joins to be equi-testable.
  • 29. Fetch models generation.
  • 30. EQL execution results instantiation.
  • 31. Enhance EQL test domain to contain all types of entities (in terms of properties, their types, calculated nature, key membership, etc.) that are actually used in final applications.
    • Alternatively, consider randomised testing (see Haskell's QuickCheck).

V. Improve troubleshooting.

  • 32. HTML representation of EQL > SQL compilation process.
  • 33. Implement QueryModel.toString() printing out models as they look in java code for more accurate error messages.
  • 34. Be able to determine that EQL query at Stage 3 contains some implicit elements (joins, yields, conditions, props, etc.).
    • would allow to equi-test cases like "no auto-yielding is done for sub-query with explicit yields".
  • 35. Be able to determine that EQL query at Stage 3 has no implicit elements (no implicit yields, joins, UDFs, conditions, etc.).
    • would allow to enforce that only such queries are used as expected models for equi-test comparison.

Notes

  • Equi-testing is a form of unit testing where SQL produced by two EQL models is expected to be the same.
    • First EQL model (model under test) is at high level of abstraction. It employs different EQL features: calculated properties, dot-notation, auto yielding, UDFs, etc.
    • Second EQL model (expected) has a low level of abstraction (SQL like).