Skip to content

Commit f9709d9

Browse files
authored
Allow a Locale to be set for formatting (#100)
The internal DataFormatter used from Apache POI allows for setting a fixed Locale to use when reading numbers, dates etc. Initially we didn't set this and thus it always used the default Locale as set from the Java runtime. This could lead to unexpected results. We now allow to set the userLocale property and use this to configure the DataFormatter used internally. We also took the opportunity to make this work with the streaming ItemReader as well as the regular ItemReader. The streaming item reader now also uses a pre-configured DataFormatter. Closes: #98
1 parent bc7fda3 commit f9709d9

File tree

7 files changed

+143
-82
lines changed

7 files changed

+143
-82
lines changed

spring-batch-excel/README.adoc

+32-26
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# spring-batch-excel
1+
= spring-batch-excel
22

33
Spring Batch extension containing an `ItemReader` implementation for Excel based on https://poi.apache.org[Apache POI]. It supports reading both XLS and XLSX files. For the latter, there is also (experimental) streaming support.
44

@@ -8,26 +8,28 @@ To reduce the memory footprint the `StreamingXlsxItemReader` can be used, this w
88

99
NOTE: The `ItemReader` classess are **not threadsafe**. The API from https://poi.apache.org/help/faq.html#20[Apache POI] itself isn't threadsafe as well as the https://docs.spring.io/spring-batch/docs/current/api/org/springframework/batch/item/support/AbstractItemCountingItemStreamItemReader.html[`AbstractItemCountingItemStreamItemReader`] used as a base class for the `ItemReader` classes. Reading from multiple threads is therefore not supported. Using a multi-threaded processor/writer should work as long as you use a single thread for reading.
1010

11-
## Configuration of `PoiItemReader`
11+
== Configuration of `PoiItemReader`
1212

1313
Next to the https://docs.spring.io/spring-batch/reference/html/configureJob.html[configuration of Spring Batch] one needs to configure the `PoiItemReader`.
1414

1515
Configuration of can be done in XML or Java Config.
1616

17-
### XML
17+
=== XML
1818

19-
```xml
19+
[source,xml]
20+
----
2021
<bean id="excelReader" class="org.springframework.batch.extensions.excel.poi.PoiItemReader" scope="step">
2122
<property name="resource" value="file:/path/to/your/excel/file" />
2223
<property name="rowMapper">
2324
<bean class="org.springframework.batch.extensions.excel.mapping.PassThroughRowMapper" />
2425
</property>
2526
</bean>
26-
```
27+
----
2728

28-
### Java Config
29+
=== Java Config
2930

30-
```java
31+
[source,java]
32+
----
3133
@Bean
3234
@StepScope
3335
public PoiItemReader excelReader() {
@@ -41,43 +43,45 @@ public PoiItemReader excelReader() {
4143
public RowMapper rowMapper() {
4244
return new PassThroughRowMapper();
4345
}
44-
```
46+
----
4547

46-
## Configuration of `StreamingXlsxItemReader`
48+
== Configuration of `StreamingXlsxItemReader`
4749

4850
Configuration can be done in XML or Java Config.
4951

50-
### XML
52+
=== XML
5153

52-
```xml
54+
[source,xml]
55+
----
5356
<bean id="excelReader" class="org.springframework.batch.extensions.excel.streaming.StreamingXlsxItemReader" scope="step">
5457
<property name="resource" value="file:/path/to/your/excel/file" />
5558
<property name="rowMapper">
5659
<bean class="org.springframework.batch.extensions.excel.mapping.PassThroughRowMapper" />
5760
</property>
5861
</bean>
59-
```
62+
----
6063

61-
### Java Config
64+
=== Java Config
6265

63-
```java
66+
[source,java]
67+
----
6468
@Bean
6569
@StepScope
66-
public StreamingXlsxItemReader excelReader() {
70+
public StreamingXlsxItemReader excelReader(RowMapper rowMapper) {
6771
StreamingXlsxItemReader reader = new StreamingXlsxItemReader();
6872
reader.setResource(new FileSystemResource("/path/to/your/excel/file"));
69-
reader.setRowMapper(rowMapper());
73+
reader.setRowMapper(rowMapper);
7074
return reader;
7175
}
7276
7377
@Bean
7478
public RowMapper rowMapper() {
7579
return new PassThroughRowMapper();
7680
}
77-
```
81+
----
7882

7983

80-
## Configuration properties
84+
== Configuration properties
8185
[cols="1,1,1,4"]
8286
.Properties for item readers
8387
|===
@@ -91,28 +95,30 @@ public RowMapper rowMapper() {
9195
| `rowSetFactory` | no | `DefaultRowSetFactory` | For reading rows a `RowSet` abstraction is used. To construct a `RowSet` for the current `Sheet` a `RowSetFactory` is needed. The `DefaultRowSetFactory` constructs a `DefaultRowSet` and `DefaultRowSetMetaData`. For construction of the latter a `ColumnNameExtractor` is needed. At the moment there are 2 implementations
9296
| `skippedRowsCallback` | no | `null` | When rows are skipped an optional `RowCallbackHandler` is called with the skipped row. This comes in handy when one needs to write the skipped rows to another file or create some logging.
9397
| `strict` | no | `true` | This controls wether or not an exception is thrown if the file doesn't exists or isn't readable, by default an exception will be thrown.
94-
| `datesAsIso` | no | `false` | Controls if dates need to be parsed as ISO or to use the format as specified in the excel sheet. *NOTE:* Only for the `PoiItemReader` **not** the `StreamingXlsxReader`!
98+
| `datesAsIso` | no | `false` | Controls if dates need to be parsed as ISO or to use the format as specified in the excel sheet.
99+
| `userLocale` | no | `null` | Set the `java.util.Locale` to use when formatting dates when there is no explicit format set in the Excel document.
95100
|===
96101

97-
- `StaticColumnNameExtractor` uses a preset list of column names.
102+
- `StaticColumnNameExtractor` uses a preset list of column names.
98103
- `RowNumberColumnNameExtractor` (**the default**) reads a given row (default 0) to determine the column names of the current sheet
99104

100-
## RowMappers
105+
== RowMappers
101106
To map a read row a `RowMapper` is needed. Out-of-the-box there are 2 implementations. The `PassThroughRowMapper` and `BeanWrapperRowMapper`.
102107

103-
### PassThroughRowMapper
108+
=== PassThroughRowMapper
104109
Transforms the read row from excel into a `String[]`.
105110

106-
### BeanWrapperRowMapper
111+
=== BeanWrapperRowMapper
107112
Uses a `BeanWrapper` to convert a given row into an object. Uses the column names of the given `RowSet` to map column to properties of the `targetType` or prototype bean.
108113

109-
```java
114+
[source,xml]
115+
----
110116
<bean id="excelReader" class="org.springframework.batch.extensions.excel.poi.PoiItemReader" scope="step">
111117
<property name="resource" value="file:/path/to/your/excel/file" />
112118
<property name="rowMapper">
113119
<bean class="org.springframework.batch.extensions.excel.mapping.BeanWrapperRowMapper">
114120
<property name="targetType" value="com.your.package.Player" />
115-
<bean>
121+
</bean>
116122
</property>
117123
</bean>
118-
```
124+
----

spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/AbstractExcelItemReader.java

+36-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/*
2-
* Copyright 2006-2021 the original author or authors.
2+
* Copyright 2006-2022 the original author or authors.
33
*
44
* Licensed under the Apache License, Version 2.0 (the "License");
55
* you may not use this file except in compliance with the License.
@@ -16,8 +16,11 @@
1616

1717
package org.springframework.batch.extensions.excel;
1818

19+
import java.util.Locale;
20+
1921
import org.apache.commons.logging.Log;
2022
import org.apache.commons.logging.LogFactory;
23+
import org.apache.poi.ss.usermodel.DataFormatter;
2124

2225
import org.springframework.batch.extensions.excel.support.rowset.DefaultRowSetFactory;
2326
import org.springframework.batch.extensions.excel.support.rowset.RowSet;
@@ -65,6 +68,12 @@ public abstract class AbstractExcelItemReader<T> extends AbstractItemCountingIte
6568

6669
private String password;
6770

71+
private boolean datesAsIso = false;
72+
73+
private Locale userLocale;
74+
75+
private DataFormatter dataFormatter;
76+
6877
public AbstractExcelItemReader() {
6978
super();
7079
this.setName(ClassUtils.getShortName(this.getClass()));
@@ -213,6 +222,16 @@ public void setResource(final Resource resource) {
213222

214223
public void afterPropertiesSet() throws Exception {
215224
Assert.notNull(this.rowMapper, "RowMapper must be set");
225+
if (this.datesAsIso) {
226+
this.dataFormatter = (this.userLocale != null) ? new IsoFormattingDateDataFormatter(this.userLocale) : new IsoFormattingDateDataFormatter();
227+
}
228+
else {
229+
this.dataFormatter = (this.userLocale != null) ? new DataFormatter(this.userLocale) : new DataFormatter();
230+
}
231+
}
232+
233+
protected DataFormatter getDataFormatter() {
234+
return this.dataFormatter;
216235
}
217236

218237
/**
@@ -295,4 +314,20 @@ public void setPassword(String password) {
295314
this.password = password;
296315
}
297316

317+
/**
318+
* Instead of using the format defined in the Excel sheet, read the date/time fields as an ISO formatted
319+
* string instead. This is by default {@code false} to leave the original behavior.
320+
* @param datesAsIso default {@code false}
321+
*/
322+
public void setDatesAsIso(boolean datesAsIso) {
323+
this.datesAsIso = datesAsIso;
324+
}
325+
326+
/**
327+
* The {@code Locale} to use when reading sheets. Defaults to the platform default as set by Java.
328+
* @param userLocale the {@code Locale} to use, default {@code null}
329+
*/
330+
public void setUserLocale(Locale userLocale) {
331+
this.userLocale = userLocale;
332+
}
298333
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
/*
2+
* Copyright 2011-2022 the original author or authors.
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* https://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
package org.springframework.batch.extensions.excel;
18+
19+
import java.time.LocalDateTime;
20+
import java.time.format.DateTimeFormatter;
21+
import java.util.Locale;
22+
23+
import org.apache.poi.ss.formula.ConditionalFormattingEvaluator;
24+
import org.apache.poi.ss.usermodel.Cell;
25+
import org.apache.poi.ss.usermodel.CellType;
26+
import org.apache.poi.ss.usermodel.DataFormatter;
27+
import org.apache.poi.ss.usermodel.DateUtil;
28+
import org.apache.poi.ss.usermodel.FormulaEvaluator;
29+
30+
/**
31+
* Specialized subclass for additionally formatting the date into an ISO date/time.
32+
*
33+
* @author Marten Deinum
34+
*
35+
* @see DateTimeFormatter#ISO_OFFSET_DATE_TIME
36+
*/
37+
public class IsoFormattingDateDataFormatter extends DataFormatter {
38+
39+
public IsoFormattingDateDataFormatter() {
40+
super();
41+
}
42+
43+
public IsoFormattingDateDataFormatter(Locale locale) {
44+
super(locale);
45+
}
46+
47+
@Override
48+
public String formatCellValue(Cell cell, FormulaEvaluator evaluator, ConditionalFormattingEvaluator cfEvaluator) {
49+
if (cell == null) {
50+
return "";
51+
}
52+
53+
CellType cellType = cell.getCellType();
54+
if (cellType == CellType.FORMULA) {
55+
if (evaluator == null) {
56+
return cell.getCellFormula();
57+
}
58+
cellType = evaluator.evaluateFormulaCell(cell);
59+
}
60+
61+
if (cellType == CellType.NUMERIC && DateUtil.isCellDateFormatted(cell, cfEvaluator)) {
62+
LocalDateTime value = cell.getLocalDateTimeCellValue();
63+
return (value != null) ? value.format(DateTimeFormatter.ISO_OFFSET_DATE_TIME) : "";
64+
}
65+
return super.formatCellValue(cell, evaluator, cfEvaluator);
66+
}
67+
}

spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/PoiItemReader.java

+1-11
Original file line numberDiff line numberDiff line change
@@ -44,11 +44,9 @@ public class PoiItemReader<T> extends AbstractExcelItemReader<T> {
4444

4545
private InputStream inputStream;
4646

47-
private boolean datesAsIso = false;
48-
4947
@Override
5048
protected Sheet getSheet(final int sheet) {
51-
return new PoiSheet(this.workbook.getSheetAt(sheet), this.datesAsIso);
49+
return new PoiSheet(this.workbook.getSheetAt(sheet), getDataFormatter());
5250
}
5351

5452
@Override
@@ -92,12 +90,4 @@ protected void openExcelFile(final Resource resource, String password) throws Ex
9290
this.workbook.setMissingCellPolicy(Row.MissingCellPolicy.CREATE_NULL_AS_BLANK);
9391
}
9492

95-
/**
96-
* Instead of using the format defined in the Excel sheet, read the date/time fields as an ISO formatted
97-
* string instead. This is by default {@code false} to leave the original behavior.
98-
* @param datesAsIso default {@code false}
99-
*/
100-
public void setDatesAsIso(boolean datesAsIso) {
101-
this.datesAsIso = datesAsIso;
102-
}
10393
}

spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/PoiSheet.java

+3-41
Original file line numberDiff line numberDiff line change
@@ -16,17 +16,13 @@
1616

1717
package org.springframework.batch.extensions.excel.poi;
1818

19-
import java.time.LocalDateTime;
20-
import java.time.format.DateTimeFormatter;
2119
import java.util.Iterator;
2220
import java.util.LinkedList;
2321
import java.util.List;
2422

25-
import org.apache.poi.ss.formula.ConditionalFormattingEvaluator;
2623
import org.apache.poi.ss.usermodel.Cell;
2724
import org.apache.poi.ss.usermodel.CellType;
2825
import org.apache.poi.ss.usermodel.DataFormatter;
29-
import org.apache.poi.ss.usermodel.DateUtil;
3026
import org.apache.poi.ss.usermodel.FormulaEvaluator;
3127
import org.apache.poi.ss.usermodel.Row;
3228

@@ -42,28 +38,23 @@
4238
class PoiSheet implements Sheet {
4339

4440
private final DataFormatter dataFormatter;
45-
4641
private final org.apache.poi.ss.usermodel.Sheet delegate;
47-
private final boolean datesAsIso;
48-
4942
private final int numberOfRows;
50-
5143
private final String name;
5244

5345
private FormulaEvaluator evaluator;
5446

5547
/**
5648
* Constructor which takes the delegate sheet.
5749
* @param delegate the apache POI sheet
58-
* @param datesAsIso should we format the dates as ISO or use the Excel formatting instead
50+
* @param dataFormatter the {@code DataFormatter} to use.
5951
*/
60-
PoiSheet(final org.apache.poi.ss.usermodel.Sheet delegate, boolean datesAsIso) {
52+
PoiSheet(final org.apache.poi.ss.usermodel.Sheet delegate, DataFormatter dataFormatter) {
6153
super();
6254
this.delegate = delegate;
63-
this.datesAsIso = datesAsIso;
6455
this.numberOfRows = this.delegate.getLastRowNum() + 1;
6556
this.name = this.delegate.getSheetName();
66-
this.dataFormatter = this.datesAsIso ? new IsoFormattingDateDataFormatter() : new DataFormatter();
57+
this.dataFormatter = dataFormatter;
6758
}
6859

6960
/**
@@ -142,33 +133,4 @@ public String[] next() {
142133
};
143134
}
144135

145-
/**
146-
* Specialized subclass for additionally formatting the date into an ISO date/time.
147-
*
148-
* @author Marten Deinum
149-
* @see DateTimeFormatter#ISO_OFFSET_DATE_TIME
150-
*/
151-
private static class IsoFormattingDateDataFormatter extends DataFormatter {
152-
153-
@Override
154-
public String formatCellValue(Cell cell, FormulaEvaluator evaluator, ConditionalFormattingEvaluator cfEvaluator) {
155-
if (cell == null) {
156-
return "";
157-
}
158-
159-
CellType cellType = cell.getCellType();
160-
if (cellType == CellType.FORMULA) {
161-
if (evaluator == null) {
162-
return cell.getCellFormula();
163-
}
164-
cellType = evaluator.evaluateFormulaCell(cell);
165-
}
166-
167-
if (cellType == CellType.NUMERIC && DateUtil.isCellDateFormatted(cell, cfEvaluator)) {
168-
LocalDateTime value = cell.getLocalDateTimeCellValue();
169-
return (value != null) ? value.format(DateTimeFormatter.ISO_OFFSET_DATE_TIME) : "";
170-
}
171-
return super.formatCellValue(cell, evaluator, cfEvaluator);
172-
}
173-
}
174136
}

0 commit comments

Comments
 (0)