lessons - isel-leic-ave/2025-lae-42d-44d GitHub Wiki

Lessons:

Lesson 01 - Introduction and Modern VMs; Class Loader

  • Bibliography and Lecturing methodology: github and slack.
  • Tools: javac, javap, kotlinc, JDK 21 and gradle.
  • Program outline in 3 parts:
    1. Java Type System and Reflection;
    2. Metaprogramming and Performance;
    3. Lazy processing.
  • Project in 3 parts according to program outline.
  • Managed Runtime or Execution Environment or informally virtual machine (VM) or runtime.
  • Execution Environment includes:
    • Compiler,
    • Programming languages,
    • Standard libraries,
    • Dependency manager (e.g. gradle, maven)
    • Central Repository (e.g. Maven Central Repository, Nuget, NPM, etc)
  • Examples of languages targeting JVM: Java, kotlin, Scala, Clojure.
  • Examples of languages targeting Node: JavaScript, TypeScript, Kotlin.
  • JVM runs .class files with bytecode
  • JVM translates bytecode to machine code (e.g. IA-32, AMD64, etc depending of the CPU)
  • In Javascript ecosystem modules are deployed in source code i.e. Javascript.
  • Distinguish between Programming Language <versus> VM
  • .class = Unit of SW distribution in JVM
  • One file .class for each class definition
  • Class Loader
  • Delayed Loading
  • Just-in-time Compiler

Lesson 02 - Reflect Intro

  • Reflection object oriented API for metadata
  • Reflection ---> metadata ---> Type System
  • Type System: types have members
  • Kotlin Reflection API: KClass ----->* KCallable
    • An instance of KClass may represent a type in Kotlin.
    • An instance of KCallable may represent a member in Kotlin.
  • KCallable base type of KFunction and KProperty
  • KProperty and KMutableProperty
  • KCallable ----->* KParameter
  • KFunction properties: name, type, parameters and instanceParameter.
  • KParameter property kind: INSTANCE versus EXTENSION_RECEIVER
  • KParameter property isOptional
  • KClass::createInstance()
  • KFunction::call()

Lesson 03 - Reflect in Java; Logger;

  • Java Reflect API;
  • Simple Logger both using Kotlin and Java Reflect API;

Lesson 04 - Reflect; Annotations; e.g. Object Mapper

  • NaiveMapper, takes inspiration from libraries like AutoMapper or MapStruct:
    • Simplify the process of mapping data between objects of different types by copying values from properties of one object to corresponding properties of another object.
    • Offers an extension function like Any.mapTo(dest: KClass<*>): Any
  • 1st version - through mutable properties (i.e. KMutableProperty):
    • The destination type must have a parameterless constructor
    • The source and destination properties share the same name and type
    • The destination properties are mutable
  • 2nd version - Match properties with different name.
  • Annotations in the JVM are a form of metadata that can be added to Java classes, methods, fields, and other program elements.
  • Annotations are strongly typed
  • Each annotation inherits from java.lang.annotation.Annotation
  • E.g. JUnit annotation @Test corresponds to the following type:
public interface org.junit.Test extends java.lang.annotation.Annotation{...}
  • Kotlin Reflect API on annotations:
    • annotations: List<Annotation>
    • findAnnotation<T>(): T
    • hasAnnotation<T>(): Bool
  • When a Kotlin member generates multiple Java members, there are multiple potential locations.
  • Use site target to explicitly specify the destination location within the metadata:
    • e.g. @property:MapProp("from") val country: String
  • Specify the allowed elements with the @Target annotation.
  • Enhance NaiveMapper to map properties to parameters with different name through the annotation @Match

Lesson 08 - Chapter 4 - Performance testing and Microbenchmarking

  • Benchmark - assess the relative performance
  • Benchmark != Unit Tests
  • A naif approach - direct measurement, e.g.
    • measureTimeMillis { System.out.log(Rectangle(...)) }.also { dur -> println("Logging rectangle takes $dur ms"); }
  • Some Problems:
    1. IDE (e.g. InteliJ) may induce other overheads.
    2. Mixing domain instantiation (i.e. Rectangle) with operation execution log().
    3. First execution includes Jitter overhead and misses optimizations.
    4. IO may be orders of magnitude slower than log operation itself
    5. Milliseconds could not be accurate enough.
    6. System.currentTimeMillis() includes a System call with implicit overhead.
    7. Garbage Collector may degrade performance
    8. Absolute performance analysis Leads to bias
  • Minimize side effects:
    1. Avoid extra tools such as IDE (e.g. InteliJ), gradle, or other => run directly on VM (e.g. java)
    2. Remove domain instantiation from operation measurement
    3. Include warm-up => Optimizations may improve performance
    4. Avoid IO => Mocking IO
    5. Measure the total execution of several iterations rather than several measurements of single executions.
    6. same as 6.
    7. Run several iterations and discard most divergent results.
    8. Baseline => Use a reference comparison to estimate how much we can improve performance.
  • Comparing performance between a:
    1. Reflection-based mem.log(Rectangle(...))
    2. Baseline ad-hoc rect.log(mem)
  • Reflection-based approach experiences a slowdown of around 5x:
Bench Log via Reflect takes  ~300 nanos
Bench Log via baseline takes  ~60 nanos

Lesson 09 - Chapter 3.3 - Minimizing Reflection Usage

  • Comparing performance between a:
    1. Reflection-based dto.mapTo(Person::class)
    2. Baseline ad-hoc dto.toPerson()
  • Reflection-based approach experiences a slowdown of around 140x:
Bench Reflect mapTo(Person::class) takes  ~700 nanos
Bench baseline toPerson() takes             ~5 nanos
  • New NaiveMapper that stores information about matching KProperty instances.
  • Approach: Look once for matching properties (1.) and use many times copying values (2.):
    1. val props: Map<KProperty<*>, KMutableProperty<*>?>
    2. props.forEach({ (from, to) -> to?.setter?.call(target, from.call(src)) })
  • Comparing performance improves slowdown of Reflection-based to 34x:
Bench Reflect mapTo(Person::class) takes     ~700 nanos
Bench Reflect NaiveMapper.mapFrom(dto) takes ~170 nanos
Bench baseline toPerson() takes                ~5 nanos

Lesson 10 - Chapter 3.5 - Reflection of generic types

  • Enhance NaiveMapper:
    1. The destination properties can be immutable and the destination type does not require a parameterless constructor
    2. The source and destination properties may have different types when they are of domain class types.
  • KCallable:
    • call(vargars) - arguments should be passed in the same order as the formal parameters of the function.
    • callBy(Map<KParameter, Any?>) - the positional order of the arguments is determined by the information of each KParameter.
  • To directly reference a KType, we use the typeOf function:
    • e.g. func.returnType != typeOf<Unit>()
  • KType holds information about nullability and type arguments.
  • KType properties: isMarkedNullable, arguments, and classifier.
    • arguments provide information about the type arguments (i.e. List<KType>)
    • classifier provides a reference to the associated class (i.e. KClassifier).
      • KClassifier is the base type of KClass

Lesson 11 - Enhance mapFrom(); JMH framework;

  • Enhance mapFrom() to suppress the overhead of type checking:
    • i.e. if (srcProp.returnType != ctorParam.type) { ... }
  • Check types on mapper initialization and manage 2 different versions of a property mapper functions (i.e. (Any?) -> Any?)):
    1. If same type - { propValue -> propValue } (identity function)
    2. For different types - { propValue -> propertyMapper.mapFrom(propValue) }
  • Store the property mapper function (i.e. (Any?) -> Any?)) in the props data structure:
    • props: List<PropInfo>
    • class PropInfo(val srcProp: KProperty<*>, val ctorProp: KParameter, val mapPropValue: (Any?) -> Any?)
  • Enhanced mapFrom:
fun mapFrom(src: T): R = props
        .associate { (srcProp, ctorParam, mapPropValue) ->
            val propValue = srcProp.call(src)
            ctorParam to mapPropValue(propValue)
        }
        .let { params: Map<KParameter, Any?> ->
            constructor.callBy(params)
        }
  • JMH - Java Microbenchmark Harness.
  • Benchmark tests annotated with @Benchmark.
  • JMH Gradle Plugin
  • gradlew jmhJar
  • java -jar <path to JAR> -f 1 -wi 4 -i 4 -w 2 -r 2 -tu ms:
    • -f - forks
    • -wi - warm-up iterations
    • -i - iterations
    • -w - each warm-up iteration duration
    • -r - each iteration duration
    • -tu - time unit

Lesson 12 - Chapter 2.1 - Java Type System

  • Type System - Set of rules and principles that specify how types are defined and behave.
  • Two kinds of types: Primitive and Reference types.
  • Classes have Members
  • Members may be: Fields or Methods.
  • There are NO properties in Java Type System.
  • Using javap -p Rectangle.class to inspect metadata
  • The fully qualified name of a class includes its package name.
  • Constructor is a method with the name <init> returning void.
  • Member access syntax: Receiver.Member.
  • The Receiver is the target of the member access and it is a Type (for static members) or an Object (for non-static/instance members).
  • JOL - Java Object Layout:
    • java -cp .;jol-cli-0.17-full.jar org.openjdk.jol.Main estimates <classqualifiedname>
    • (linux replace ; by : on -cp (classpath))
  • Object header = mark word (used for hash, locks, GC, etc) + class word (class-specific metadata).
  • Fields alignment, reorder and padding gap.

Lesson 13 - Chapter 2.2, 2.3, 2.5 - Abstract and Base Types; Anonymous classes; Function Types

  • Nested Types
  • Abstract and Base Types
  • Anonymous classes
  • Function Types

Lesson 14 - Object declarations (Chap 2.4.2); Boxing and Unboxing (Chap 2.1.4)

  • Singleton design pattern
  • object keyword:
    • private constructor
    • Singleton instance in static INSTANCE field;
  • Static initializer: initializes static fields (i.e. <cinit>())
  • companion object - specific type of object declaration associated with its owner class.
  • Boxing and Unboxing.
  • Categories of Types:
    • Value Types (e.g. Primitive types, e.g. Java int, long, double,...)
    • Reference Types (e.g. class, interface,...)
  • Values <versus> Objects (in Heap);
  • Primitive <versus> Reference types
  • int, long, double, ... <versus> Integer, Long, Double, ...
  • Autoboxing is the automatic conversion that the Java compiler makes between the primitive types and their corresponding object wrapper classes.
  • unboxing - converting an object of a wrapper type (e.g. Integer) to its corresponding primitive (int) value is called .
  • Primitive <-> Reference: boxing or unboxing:
    • There is no specific JVM bytecode for these conversions.
    • Supported in JVM through auxiliary functions (i.e. valueOf() and <type>Value()) of Wrapper classes.
  • Primitive Type -> Reference: boxing through <Wrapper Type>.valueOf(primitive)
    • e.g. fun boxing(nr: Int) : Any { return nr }
  • Reference Type -> Primitive: unboxing through <Wrapper>.<primitive>Value()
    • e.g. fun unboxing(nr: Int?) : Int { return nr ?: throw Exception("Null not supported for primitive Int!") }
Types Java Kotlin
Value int, long, double, ... Int, Long, Double, ...
Reference Integer, Long, Double, ... Int?, Long?, Double?, ...

Lesson 17 - 2.2: Abstract and Base Types (Chap 2.2);

  • Names collision and Member access ambiguity
  • Methods call resolution: Fields, Methods, and Virtual Methods.
  • Override.
Methods Kotlin Java
Static Dispatch
i.e. type of the receiver 		- final
static
Dynamic Dispatch
i.e. type of the object 		open
abstract 		abstract

Homework

  • Consider the following definition of types A, B, C, and I. Given the use of those types in main, answer the following questions:
    1. Predict the output of the execution of the main function.
    2. What are the differences in resulting output if we remove the final keyword from virtualFoo in class B?
    3. Keeping the definition of types A, B, C, and I as it is, what would be the result of adding the following implementation in class C:
      • public void virtualFoo() { out.println("C"); }
    4. Provide an equivalent implementation in Kotlin for types A, B, C, and I, with only the method virtualFoo to achieve the same result as in question 1. Do not include the static method foo.
interface I { void virtualFoo(); }
class A {
    public static void foo() { out.println("A"); }
    public void virtualFoo(){ out.println("A"); }
}
class B extends A implements I {
    public static void foo() { out.println("B"); }
    public final void virtualFoo(){ out.println("B"); }
}
class C extends B {
    public static void foo(){ out.println("C"); }
}
 
public static void main(String[] args) {
    final C c = new C();
    final A a = c;
    final B b = c;
    final I i = c;
    a.foo();
    a.virtualFoo();
    b.foo();
    b.virtualFoo();
    c.foo();
    c.virtualFoo();
    i.virtualFoo();       
}

Lesson 20 - bytecode (Chap 5);

  • Evaluations Stack
  • Local variables and Arguments
  • Constant Pool
  • 16-bit index into the constant pool
  • Load and store opcodes
  • Shortcut opcode forms
  • Arithmetic
  • Execution Flow
  • Reference Types are instantiated in bytecode with:
    • new - Allocates storage on Heap, initializes space and the object's header, and returns the reference to newbie object.
    • invokespecial - Call to class <init> method (corresponding to constructor).
  • Instantiating a refence type, e.g. Student(765134, "Ze Manel") may produce in bytecode:
new           #8   // class Student
dup                // duplicates the value on top of the stack
...                // One load (push) for each parameter of <init> (constructor)
invokespecial #14  // Method Student."<init>"
  • invokestatic - no this required
  • invokespecial - static dispatch (i.e. non polymorphic)
  • invokevirtual - dynamic dispatch (i.e. polymorphic)
  • invokeinterface - dynamic dispatch (i.e. polymorphic) for interface methods

Homework

  1. Run java Foo in the lesson20-TPC folder and observe the output.
  2. Write the Java equivalent of the bar function defined in Foo.class from lesson20-TPC.
  3. Test your implementation with the arguments 12 and 19 to ensure it produces the same output as in step 1.

Lesson 21 - Metaprogramming; Class-File API;

---
config:
  theme: mc
  look: classic
  layout: dagre
---
classDiagram
direction LR
    class ClassFile {
        of() ClassFile
        build(ClassDesc, Consumer~ClassBuilder~) byte[]
    }
    class ClassBuilder {
        withFlags(int)
        withField(String, ClassDesc, int, Consumer~FieldBuilder~)
        withMethod(String, MethodTypeDesc, int, Consumer~MethodBuilder~)

    }
    class MethodBuilder {
        withCode(Consumer~CodeBuilder~)
    }
    ClassFile ..> ClassBuilder
    ClassBuilder ..> FieldBuilder
    ClassBuilder ..> MethodBuilder
Loading

Lesson 23 - NaiveMapper Dynamic

  • New Dynamic Mapper that suppresses Reflect on:
    1. Getting properties from source object
    2. Instantiating the target class
  • Common base interface for NaiveMapper and Dynamic Mappers:
    • interface Mapper<T> { fun mapFrom(source: Any): T }
  • NOTE:
    • There is a single NaiveMapper class using Reflect
    • There are different Mapper classes dynamically generated for each pair srcKlass to destKlass
  • Each Dynamic Mapper has a different implementation of mapFrom.

Java Reflect <interop> Kotlin Reflect:

  • Annotation @JvmOverloads -- Instructs the Kotlin compiler to generate overloads for a function that substitute default parameter values.
  • javac -parameters - Generates metadata for reflection on method parameters.
    • Gradle : tasks.compileKotlin { kotlinOptions { javaParameters = true } }

Lesson 28 - Sequences

  • Collection Pipeline - "organize some computation as a sequence of operations which compose by taking a collection as output of one operation and feeding it into the next."
  • Advantages:
    • Composability
    • Expressivity/Readability
    • Extensibility
  • Alternative pipelines idioms:
    • e.g. method chaining:
      • students.filter(...).map(...).distinct(...).count()
    • e.g. nested function:
      • (count(remove-duplicates(mapcar #'map-function(remove-if-not ...))))
  • Collection pipeline:
    • Data Source --> Intermediate Operation* --> Terminal Operation
    • May have many intermediate operations.
    • After the terminal operation we cannot chain any intermediate operation.
  • May distinguish by:
    • Idiom of combining functions: method chain versus nested functions
    • API methods names: e.g. filter, drop, reduce, etc versus where, skip, fold, etc (in Dart)
    • Eager versus Lazy, e.g. JavaScript Array methods versus Java java.util.stream
  • Query Example: Distinct weather descriptions in rainy days.
weatherData
  .map { it.weatherDesc }
  .filter { it.lowercase().contains("rain") }
  .distinct()
weatherData
  .filter { it.weatherDesc.lowercase().contains("rain") }
  .map { it.weatherDesc }
  .distinct()
  • The order of intermediate operations in the pipeline => may impact the total number of iterations.
  • Custom implementation of an equivalent eagerMap and eagerFilter.
  • Kotlin 2 distinct APIs for collections pipeline: Iterable versus Sequence
  • Iterable and Collections are eager with horizontal processing
  • Sequence should be lazy with vertical processing and can be infinite.
  • Marble diagrams:
    • Time/Data is horizontally from left to right.
    • Operations flow vertically, representing the sequence of operations applied to the data stream
  • Implementing lazyMap equivalent to the lazy version of map
  • Explicit implementation of interface Iteratorfor lazyMap
  • Homework:
    • Implement lazyDistinct.
      • Explicit implementation of interface Iteratorfor lazyDistinct

Lesson 29 - Generators and yield operator

  • Generator:
    • Like a function, but instead of returning a single value, it produces a sequence of values.
    • Uses the yield keyword to return the next item in the sequence to the caller.
    • Computation is suspended at yield and resumed when the caller requests the next item (next()).
  • A generator can remain suspended indefinitely without being resumed from its caller.
  • sequence - Builds a Sequence lazily yielding values one by one.
  • suspend fun yield(value: T) - Yields a value to the Iterator being built and suspends until the next value is requested.

Homework -- Exercises with lazy sequences:

  1. Implementing alternative suspendDistinct in 4 lines using the yield.
  2. fun <T> Sequence<T>.lazyConcat(other: Sequence<T>): Sequence<T> with explicit implementation of interface Iterator
  3. fun <T> Sequence<T>.suspendConcat(other: Sequence<T>): Sequence<T> using yield
  4. fun <T : Any?> Sequence<T>.collapse(): Sequence<T> - merges series of adjacent elements

Lesson 30 - Exercises Sequences and Generators

  • Exercises Sequences and Generators

Lesson 31 - Exercises Sequences and Generators

  • Write an extension function suspZip for Sequence<T> that combines the receiver sequence with another sequence (other) into a new Sequence<V>.
  • The combination must be done element-wise using the provided transform function.
fun <T, R, V> Sequence<T>.suspZip(
    other: Sequence<R>,
    transform: (a: T, b: R) -> V
): Sequence<V>
  • The resulting sequence must be lazy (like zip) and should terminate when either input sequence runs out of elements.
  • The function should not materialize the sequences in memory (i.e. do not convert to lists/arrays).
  • This function is not suspendable, despite the name
val numbers = sequenceOf(1, 2, 3)
val letters = sequenceOf("a", "b", "c", "d")

val zipped = numbers.suspZip(letters) { num, letter -> "$num-$letter" }
println(zipped.toList()) // Output: [1-a, 2-b, 3-c]

Lesson 32 - Deconstructing yield and Suspend Functions

  • Continuation<T>
  • resumeWith(result: Result<Unit>)
  • Suspension points and State Machine
  • Example cast fetchSuspend(String) in fetchCps(url: String, onComplete: Continuation<String>): Any
  • Example cast fetchCps(url: String, onComplete: Continuation<String>): Any to:
    • suspend (String) -> String

Lesson 34 - Garbage Collector

  • Local values in Stack <versus> Objects in Heap.
  • Local variables managed on stack <versus> Objects managed on Heap by Garbage Collector
    • Stack frame on function's execution => cleaned on function completion, including local variables.
    • Unreachable objects are candidates for Garbage Collector
  • Garbage Collection - "process of looking at heap memory, identifying which objects are in use and which are not, and deleting the unused objects."
  • Unused object, or unreferenced object.
  • In use object, or a referenced object:
    • "some part of your program still maintains a pointer to that object."
    • Referenced by a root or belongs to a graph with a root reference.
    • root reference:
      • Local variables - stored in the stack of a thread.
      • Static variables - belong to the class type stored in permanent generation.
  • GC basic process: 1. Marking and 2. Deletion.
  • Deletion approaches:
    • Normal Deletion => holds a list to free spaces. !! memory allocation slower !!
    • Deletion with Compacting => move referenced objects together ++ makes new memory allocation faster !!! longer garbage collection time !!!
  • Generational Garbage Collection:
    • Most objects are short lived => GC is more effective in younger generations.
    • Enhance the performance
  • JVM Generations:
    • Young Generation is where all new objects are allocated and aged.
    • Old Generation is used to store long surviving objects.
    • Permanent generation contains metadata, classes and methods.
  • "Stop the World" events:
    • minor garbage collection - collects the young generation.
    • major garbage collection - collects the old generation.
  • Young Generation process: Eden, Survivor Space 0 and Survivor Space 1.

Lesson 36 - Closeable

Lesson 37 - Cleaner

  • Cleaner - manage cleaning actions.
    • public Cleanable register(Object obj, Runnable action) - Registers an object and a cleaning action, i.e. Runnable
    • Cleanable has a single method clean() that runs the cleaning action and unregisters the cleanable.
  • Explicitly invoke the clean() method when the object is closed or no longer needed.
  • If the close() method is not called, the cleaning action is called by the Cleaner.
⚠️ **GitHub.com Fallback** ⚠️