Skip to content

Commit 9538c78

Browse files
authored
docs: Collection pipeline explanation (iluwatar#2875)
* collection pipeline docs + refactoring * restore imperative programming code
1 parent 7c1889b commit 9538c78

File tree

5 files changed

+148
-47
lines changed

5 files changed

+148
-47
lines changed

collection-pipeline/README.md

Lines changed: 128 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,25 +3,145 @@ title: Collection Pipeline
33
category: Functional
44
language: en
55
tag:
6-
- Reactive
6+
- Reactive
7+
- Data processing
78
---
89

910
## Intent
10-
Collection Pipeline introduces Function Composition and Collection Pipeline, two functional-style patterns that you can combine to iterate collections in your code.
11-
In functional programming, it's common to sequence complex operations through a series of smaller modular functions or operations. The series is called a composition of functions, or a function composition. When a collection of data flows through a function composition, it becomes a collection pipeline. Function Composition and Collection Pipeline are two design patterns frequently used in functional-style programming.
11+
12+
The Collection Pipeline design pattern is intended to process collections of data by chaining together operations in a
13+
sequence where the output of one operation is the input for the next. It promotes a declarative approach to handling
14+
collections, focusing on what should be done rather than how.
15+
16+
## Explanation
17+
18+
Real-world example
19+
20+
> Imagine you're in a large library filled with books, and you're tasked with finding all the science fiction books
21+
> published after 2000, then arranging them by author name in alphabetical order, and finally picking out the top 5 based
22+
> on their popularity or ratings.
23+
24+
In plain words
25+
26+
> The Collection Pipeline pattern involves processing data by passing it through a series of operations, each
27+
> transforming the data in sequence, much like an assembly line in a factory.
28+
29+
Wikipedia says
30+
31+
> In software engineering, a pipeline consists of a chain of processing elements (processes, threads, coroutines,
32+
> functions, etc.), arranged so that the output of each element is the input of the next; the name is by analogy to a
33+
> physical pipeline. Usually some amount of buffering is provided between consecutive elements. The information that flows
34+
> in these pipelines is often a stream of records, bytes, or bits, and the elements of a pipeline may be called filters;
35+
> this is also called the pipe(s) and filters design pattern. Connecting elements into a pipeline is analogous to function
36+
> composition.
37+
38+
**Programmatic Example**
39+
40+
The Collection Pipeline pattern is implemented in this code example by using Java's Stream API to perform a series of
41+
transformations on a collection of Car objects. The transformations are chained together to form a pipeline. Here's a
42+
breakdown of how it's done:
43+
44+
1. Creation of Cars: A list of Car objects is created using the `CarFactory.createCars()` method.
45+
46+
`var cars = CarFactory.createCars();`
47+
48+
2. Filtering and Transforming: The `FunctionalProgramming.getModelsAfter2000(cars)` method filters the cars to only
49+
include those made after the year 2000, and then transforms the filtered cars into a list of their model names.
50+
51+
`var modelsFunctional = FunctionalProgramming.getModelsAfter2000(cars);`
52+
53+
In the `getModelsAfter2000` method, the pipeline is created as follows:
54+
55+
```java
56+
public static List<String> getModelsAfter2000(List<Car> cars){
57+
return cars.stream().filter(car->car.getYear()>2000)
58+
.sorted(comparing(Car::getYear))
59+
.map(Car::getModel)
60+
.collect(toList());
61+
}
62+
```
63+
64+
3. Grouping: The `FunctionalProgramming.getGroupingOfCarsByCategory(cars)` method groups the cars by their category.
65+
66+
`var groupingByCategoryFunctional = FunctionalProgramming.getGroupingOfCarsByCategory(cars);`
67+
68+
In the getGroupingOfCarsByCategory method, the pipeline is created as follows:
69+
70+
```java
71+
public static Map<Category, List<Car>>getGroupingOfCarsByCategory(List<Car> cars){
72+
return cars.stream().collect(groupingBy(Car::getCategory));
73+
}
74+
```
75+
76+
4. Filtering, Sorting and Transforming: The `FunctionalProgramming.getSedanCarsOwnedSortedByDate(List.of(john))` method
77+
filters the cars owned by a person to only include sedans, sorts them by date, and then transforms the sorted cars
78+
into a list of Car objects.
79+
80+
`var sedansOwnedFunctional = FunctionalProgramming.getSedanCarsOwnedSortedByDate(List.of(john));`
81+
82+
In the `getSedanCarsOwnedSortedByDate` method, the pipeline is created as follows:
83+
84+
```java
85+
public static List<Car> getSedanCarsOwnedSortedByDate(List<Person> persons){
86+
return persons.stream().flatMap(person->person.getCars().stream())
87+
.filter(car->Category.SEDAN.equals(car.getCategory()))
88+
.sorted(comparing(Car::getDate))
89+
.collect(toList());
90+
}
91+
```
92+
93+
In each of these methods, the Collection Pipeline pattern is used to perform a series of operations on the collection of
94+
cars in a declarative manner, which improves readability and maintainability.
1295

1396
## Class diagram
97+
1498
![alt text](./etc/collection-pipeline.png "Collection Pipeline")
1599

16100
## Applicability
17-
Use the Collection Pipeline pattern when
18101

19-
* When you want to perform a sequence of operations where one operation's collected output is fed into the next
20-
* When you use a lot of statements in your code
21-
* When you use a lot of loops in your code
102+
This pattern is applicable in scenarios involving bulk data operations such as filtering, mapping, sorting, or reducing
103+
collections. It's particularly useful in data analysis, transformation tasks, and where a sequence of operations needs
104+
to be applied to each element of a collection.
105+
106+
## Known Uses
107+
108+
* LINQ in .NET
109+
* Stream API in Java 8+
110+
* Collections in modern functional languages (e.g., Haskell, Scala)
111+
* Database query builders and ORM frameworks
112+
113+
## Consequences
114+
115+
Benefits:
116+
117+
* Readability: The code is more readable and declarative, making it easier to understand the sequence of operations.
118+
* Maintainability: Easier to modify or extend the pipeline with additional operations.
119+
* Reusability: Common operations can be abstracted into reusable functions.
120+
* Lazy Evaluation: Some implementations allow for operations to be lazily evaluated, improving performance.
121+
122+
Trade-offs:
123+
124+
* Performance Overhead: Chaining multiple operations can introduce overhead compared to traditional loops, especially
125+
for short pipelines or very large collections.
126+
* Debugging Difficulty: Debugging a chain of operations might be more challenging due to the lack of intermediate
127+
variables.
128+
* Limited to Collections: Primarily focused on collections, and its utility might be limited outside of collection
129+
processing.
130+
131+
## Related Patterns
132+
133+
* [Builder](https://java-design-patterns.com/patterns/builder/): Similar fluent interface style but used for object
134+
construction.
135+
* [Chain of Responsibility](https://java-design-patterns.com/patterns/chain-of-responsibility/): Conceptually similar in
136+
chaining handlers, but applied to object requests rather than data collection processing.
137+
* [Strategy](https://java-design-patterns.com/patterns/strategy/): Can be used within a pipeline stage to encapsulate
138+
different algorithms that can be selected at runtime.
22139

23140
## Credits
24141

25142
* [Function composition and the Collection Pipeline pattern](https://www.ibm.com/developerworks/library/j-java8idioms2/index.html)
26-
* [Martin Fowler](https://martinfowler.com/articles/collection-pipeline/)
143+
* [Collection Pipeline described by Martin Fowler](https://martinfowler.com/articles/collection-pipeline/)
27144
* [Java8 Streams](https://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html)
145+
* [Refactoring: Improving the Design of Existing Code](https://amzn.to/3VDMWDO)
146+
* [Functional Programming in Scala](https://amzn.to/4cEo6K2)
147+
* [Java 8 in Action: Lambdas, Streams, and functional-style programming](https://amzn.to/3THp4wy)

collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Car.java

Lines changed: 3 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -22,22 +22,11 @@
2222
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
2323
* THE SOFTWARE.
2424
*/
25-
package com.iluwatar.collectionpipeline;
2625

27-
import lombok.EqualsAndHashCode;
28-
import lombok.Getter;
29-
import lombok.RequiredArgsConstructor;
26+
package com.iluwatar.collectionpipeline;
3027

3128
/**
3229
* A Car class that has the properties of make, model, year and category.
3330
*/
34-
@Getter
35-
@EqualsAndHashCode
36-
@RequiredArgsConstructor
37-
public class Car {
38-
private final String make;
39-
private final String model;
40-
private final int year;
41-
private final Category category;
42-
43-
}
31+
public record Car(String make, String model, int year, Category category) {
32+
}

collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/FunctionalProgramming.java

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
2323
* THE SOFTWARE.
2424
*/
25+
2526
package com.iluwatar.collectionpipeline;
2627

2728
import java.util.Comparator;
@@ -54,9 +55,8 @@ private FunctionalProgramming() {
5455
* @return {@link List} of {@link String} representing models built after year 2000
5556
*/
5657
public static List<String> getModelsAfter2000(List<Car> cars) {
57-
return cars.stream().filter(car -> car.getYear() > 2000)
58-
.sorted(Comparator.comparing(Car::getYear))
59-
.map(Car::getModel).toList();
58+
return cars.stream().filter(car -> car.year() > 2000).sorted(Comparator.comparing(Car::year))
59+
.map(Car::model).toList();
6060
}
6161

6262
/**
@@ -66,7 +66,7 @@ public static List<String> getModelsAfter2000(List<Car> cars) {
6666
* @return {@link Map} with category as key and cars belonging to that category as value
6767
*/
6868
public static Map<Category, List<Car>> getGroupingOfCarsByCategory(List<Car> cars) {
69-
return cars.stream().collect(Collectors.groupingBy(Car::getCategory));
69+
return cars.stream().collect(Collectors.groupingBy(Car::category));
7070
}
7171

7272
/**
@@ -76,8 +76,8 @@ public static Map<Category, List<Car>> getGroupingOfCarsByCategory(List<Car> car
7676
* @return {@link List} of {@link Car} to belonging to the group
7777
*/
7878
public static List<Car> getSedanCarsOwnedSortedByDate(List<Person> persons) {
79-
return persons.stream().map(Person::getCars).flatMap(List::stream)
80-
.filter(car -> Category.SEDAN.equals(car.getCategory()))
81-
.sorted(Comparator.comparing(Car::getYear)).toList();
79+
return persons.stream().map(Person::cars).flatMap(List::stream)
80+
.filter(car -> Category.SEDAN.equals(car.category()))
81+
.sorted(Comparator.comparing(Car::year)).toList();
8282
}
8383
}

collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -61,21 +61,21 @@ public static List<String> getModelsAfter2000(List<Car> cars) {
6161
List<Car> carsSortedByYear = new ArrayList<>();
6262

6363
for (Car car : cars) {
64-
if (car.getYear() > 2000) {
64+
if (car.year() > 2000) {
6565
carsSortedByYear.add(car);
6666
}
6767
}
6868

6969
Collections.sort(carsSortedByYear, new Comparator<Car>() {
7070
@Override
7171
public int compare(Car car1, Car car2) {
72-
return car1.getYear() - car2.getYear();
72+
return car1.year() - car2.year();
7373
}
7474
});
7575

7676
List<String> models = new ArrayList<>();
7777
for (Car car : carsSortedByYear) {
78-
models.add(car.getModel());
78+
models.add(car.model());
7979
}
8080

8181
return models;
@@ -90,12 +90,12 @@ public int compare(Car car1, Car car2) {
9090
public static Map<Category, List<Car>> getGroupingOfCarsByCategory(List<Car> cars) {
9191
Map<Category, List<Car>> groupingByCategory = new HashMap<>();
9292
for (Car car : cars) {
93-
if (groupingByCategory.containsKey(car.getCategory())) {
94-
groupingByCategory.get(car.getCategory()).add(car);
93+
if (groupingByCategory.containsKey(car.category())) {
94+
groupingByCategory.get(car.category()).add(car);
9595
} else {
9696
List<Car> categoryCars = new ArrayList<>();
9797
categoryCars.add(car);
98-
groupingByCategory.put(car.getCategory(), categoryCars);
98+
groupingByCategory.put(car.category(), categoryCars);
9999
}
100100
}
101101
return groupingByCategory;
@@ -111,20 +111,20 @@ public static Map<Category, List<Car>> getGroupingOfCarsByCategory(List<Car> car
111111
public static List<Car> getSedanCarsOwnedSortedByDate(List<Person> persons) {
112112
List<Car> cars = new ArrayList<>();
113113
for (Person person : persons) {
114-
cars.addAll(person.getCars());
114+
cars.addAll(person.cars());
115115
}
116116

117117
List<Car> sedanCars = new ArrayList<>();
118118
for (Car car : cars) {
119-
if (Category.SEDAN.equals(car.getCategory())) {
119+
if (Category.SEDAN.equals(car.category())) {
120120
sedanCars.add(car);
121121
}
122122
}
123123

124124
sedanCars.sort(new Comparator<Car>() {
125125
@Override
126126
public int compare(Car o1, Car o2) {
127-
return o1.getYear() - o2.getYear();
127+
return o1.year() - o2.year();
128128
}
129129
});
130130

collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Person.java

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -25,16 +25,8 @@
2525
package com.iluwatar.collectionpipeline;
2626

2727
import java.util.List;
28-
import lombok.Getter;
29-
import lombok.RequiredArgsConstructor;
3028

3129
/**
3230
* A Person class that has the list of cars that the person owns and use.
3331
*/
34-
@Getter
35-
@RequiredArgsConstructor
36-
public class Person {
37-
38-
private final List<Car> cars;
39-
40-
}
32+
public record Person(List<Car> cars) {}

0 commit comments

Comments
 (0)