Efficient JSON file processing is important in modern applications. If JSON file is large in size, we need to handle it properly to avoid heap out of memory error and heavy resource consumption.
GSON is one of the best library available in Java to process JSON files.
It provides thread safe instance to reuse the same object to process multiple requests.
It doesn't expect any annotations to be available to serialization and deserialization unless there is a special case.
It provides toJson() and fromJson() methods for serialization and deserialization.
It provides support to read the content using stream readers.
Below is the code snippet to read the file using java stream readers and convert to Java object. I am applying my business logic for every 100 records.
Assumption JSON file is having array of data like below.
[
{
"id": 1,
"firstName": "Emily",
"lastName": "Johnson"
}
]
Java Code
private Optional<Boolean> readFile(String path) {
try (InputStream inputStream = Files.newInputStream(Path.of(path));
JsonReader reader = new JsonReader(new InputStreamReader(inputStream))) {
Instant start = Instant.now();
reader.beginArray();
List<User> users = new ArrayList<>();
while (reader.hasNext()) {
users.add(gson.fromJson(reader, User.class));
if (users.size() >= 100) {
//Initiate your records processing
//You can consider processing it async way handling failure scenarios properly
users.clear();
}
}
reader.endArray();
LOGGER.info("Total time taken to process file:{}, Duration:{} (millis)", path, Duration.between(start, Instant.now()).toMillis());
} catch (Exception e) {
LOGGER.error("", e);
return Optional.of(false);
}
return Optional.of(true);
}
The above is the detailed content of Large JSON file processing in Java. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
Java Features for Modern Development: A Practical Overview
May 08, 2025 am 12:12 AM
Javastandsoutinmoderndevelopmentduetoitsrobustfeatureslikelambdaexpressions,streams,andenhancedconcurrencysupport.1)Lambdaexpressionssimplifyfunctionalprogramming,makingcodemoreconciseandreadable.2)Streamsenableefficientdataprocessingwithoperationsli
Java Platform Independence: Advantages for web applications
May 09, 2025 am 12:08 AM
Java'splatformindependencebenefitswebapplicationsbyallowingcodetorunonanysystemwithaJVM,simplifyingdeploymentandscaling.Itenables:1)easydeploymentacrossdifferentservers,2)seamlessscalingacrosscloudplatforms,and3)consistentdevelopmenttodeploymentproce
Can Java be run everywhere?
May 07, 2025 pm 06:41 PM
Yes,Javacanruneverywhereduetoits"WriteOnce,RunAnywhere"philosophy.1)Javacodeiscompiledintoplatform-independentbytecode.2)TheJavaVirtualMachine(JVM)interpretsorcompilesthisbytecodeintomachine-specificinstructionsatruntime,allowingthesameJava
The Truth About Java's Platform Independence: Is It Really That Simple?
May 09, 2025 am 12:10 AM
Java'splatformindependenceisnotsimple;itinvolvescomplexities.1)JVMcompatibilitymustbeensuredacrossplatforms.2)Nativelibrariesandsystemcallsneedcarefulhandling.3)Dependenciesandlibrariesrequirecross-platformcompatibility.4)Performanceoptimizationacros
What are the four categories of Java? Description of Java Basic Type System Classification
May 20, 2025 pm 08:27 PM
Java's four basic type systems include integer types, floating point types, character types and boolean types. 1. Integer types (byte, short, int, long) are used to store numerical values ??without decimals. Choosing the appropriate type can optimize memory and performance. 2. Float type (float, double) is used for decimal values. Pay attention to accuracy issues. If necessary, BigDecimal is used. 3. Character type (char) is based on Unicode and is suitable for single characters, but String may be required in international applications. 4. Boolean types are used for true and false values, simplifying logical judgments and improving code readability.
Java Platform Independence: What does it mean for developers?
May 08, 2025 am 12:27 AM
Java'splatformindependencemeansdeveloperscanwritecodeonceandrunitonanydevicewithoutrecompiling.ThisisachievedthroughtheJavaVirtualMachine(JVM),whichtranslatesbytecodeintomachine-specificinstructions,allowinguniversalcompatibilityacrossplatforms.Howev
Java features: a quick guide
May 07, 2025 pm 05:17 PM
Key features of Java include: 1) object-oriented design, 2) platform independence, 3) garbage collection mechanism, 4) rich libraries and frameworks, 5) concurrency support, 6) exception handling, 7) continuous evolution. These features of Java make it a powerful tool for developing efficient and maintainable software.
What is the difference between JDK and JVM?
May 07, 2025 pm 05:21 PM
JDKincludestoolsfordevelopingandcompilingJavacode,whileJVMrunsthecompiledbytecode.1)JDKcontainsJRE,compiler,andutilities.2)JVMmanagesbytecodeexecutionandsupports"writeonce,runanywhere."3)UseJDKfordevelopmentandJREforrunningapplications.
