Python read file by blocks

Read File in Python

In this article, we’ll learn how to read files in Python.

In Python, temporary data that is locally used in a module will be stored in a variable. In large volumes of data, a file is used such as text and CSV files and there are methods in Python to read or write data in those files.

After reading this tutorial, you’ll learn: –

• Reading both text and binary files
• The different modes for reading the file
• All methods for reading a text file such as read() , readline() , and readlines()
• Read text file line by line
• Read and write files at the same time.

Table of contents

Access Modes for Reading a file

To read the contents of a file, we have to open a file in reading mode. Open a file using the built-in function called open() . In addition to the file name, we need to pass the file mode specifying the purpose of opening the file.

The following are the different modes for reading the file. We will see each one by one.

# read file with absolute path try: fp = open(r"E:\demos\files\read_demo.txt", "r") print(fp.read()) fp.close() except FileNotFoundError: print("Please check the path")

First line Second line Third line Fourth line Fifth line

An absolute path contains the entire path to the file or directory that we need to access. It includes the complete directory list required to locate the file.

For example, E:\PYnative\files_demos\read_demo.txt is an absolute path to discover the read_demo.txt. All of the information needed to find the file is contained in the path string.

While opening a file for reading its contents we have always ensured that we are providing the correct path. In case the file not present in the provided path we will get FileNotFoundError .

We can avoid this by wrapping the file opening code in the try-except-finally block.

Reading a File Using the with Statement

We can open a file using the with statement along with the open function. The general syntax is as follows.

with open(__file__, accessmode) as f:

The following are the main advantages of opening a file using ‘with’ statement

• The with statement simplifies exception handling by encapsulating common preparation and cleanup tasks.
• This also ensures that a file is automatically closed after leaving the block.
• As the file is closed automatically it ensures that all the resources that are tied up with the file are released.

Let us see how we can the with statement to read a file.

# Reading files using 'with' with open('read_demo.txt', 'r') as file: print(file.read())

First line Second line Third line Fourth line Fifth line

File Read Methods

Python provides three different methods to read the file. We don’t have to import any module for that.. Below are the three methods

Reading File in Reverse Order

We can read the contents of the file in reverse order by using the readlines() method and then calling the reversed () method on the list to get the contents of the list in reverse order. We can then iterate over the contents of the list and print the values.

with open('readdemo.txt', 'r') as f: lines = f.readlines() for line in reversed(lines): print(line)

Fifth Line Fourth Line Third Line Second Line First Line

Reading a Binary file

Binary files are basically the ones with data in the Byte format (0’s and 1’s). This generally doesn’t contain the EOL(End of Line) so it is important to check that condition before reading the contents of the file.

We can open and read the contents of the binary file using the ‘with’ statement as below.

with open("Timezones.jpg", "rb") as f: byte_content = f.read(1) while byte_content: #Printing the contents of the file print(byte_content)

We have seen in this post how the file contents could be read using the different read methods available in Python. We also saw few simple examples to read the contents partially like first few lines or last few lines based on our requirement.

Did you find this page helpful? Let others know about it. Sharing helps me continue to create free Python resources.

About Vishal

I’m Vishal Hule, Founder of PYnative.com. I am a Python developer, and I love to write articles to help students, developers, and learners. Follow me on Twitter

Related Tutorial Topics:

Python Exercises and Quizzes

Free coding exercises and quizzes cover Python basics, data structure, data analytics, and more.

• 15+ Topic-specific Exercises and Quizzes
• Each Exercise contains 10 questions
• Each Quiz contains 12-15 MCQ

Источник

Read file block by block in python

Solution: You could use groupy to find each block, use a regex to extract the info and put the values in dicts: Output: Or without groupby as your file follows a format we just need to extract the second line after the BLOCK line: Output: To extract values you can iterate: Output: If you want a dict of dicts and to access each section from 1-n you can store as nested dicts using from 1-n as tke key: Output: will tell you exactly how many blocks you extracted. Question: I have file that has this format I want to read the file as blocks such that each MACRO to end of its name form a block in python.

Read file block by block in python

I have file that has this format

MACRO L20_FDPQ_TV1T8_1 FIXEDMASK ; CLASS CORE ; . . END L20_FDPQ_TV1T8_1 MACRO INV_20 . . END INV_20 

I want to read the file as blocks such that each MACRO to end of its name form a block in python. I tried to use this

with open(file_name, "r") as f_read: lines = f_read.readlines() num = 0 while num < ****(lines): line = lines[num] if re.search(r"MACRO\s+", line, re.IGNORECASE): macro_name = line.split()[1] while re.search(r"\s+END\s+%s"%macro_name, line, re.IGNORECASE) is None: line = lines[num + 1] #. do something. num = num+1 num +=1 

how this can be done in an efficient way?

Assumming that you cannot nest Macros , that Macros always start with "MACRO [name]" and end with "END [name]":

# read the contents of the file with open(file_name, "r") as f_read: lines = f_read.readlines() current_macro_lines = [] for line in lines: if line.startswith("MACRO"): macro_name = line.split()[1] # if line starts with END and refers to the current macro name elif line.startswith("END") and line.split()[1] == macro_name: # here the macro is complete, #put the code you want to execute when you find the end of the macro # then when you finish processing the lines, # empty them so you can accumulate the next macro current_macro_lines = [] else: # here you can accumulate the macro lines current_macro_lines.append(line) 

Python open file as utf 8 Code Example, python change defaut file read format to utf8. path in string python utf-8 decode. with open (text_path, encoding="utf8") as fp: with open (json, 'w', encoding='utf-8') as file: python load utf-8 file. python open file as utf 8. python open file encoding utf-8 with context. python open file 'utf

How to read a file block-wise in python

I am bit stuck in reading a file block-wise , and facing difficulty in getting some selective data in each block :

DATA.txt #-----FILE-----STARTS-----HERE--# #--COMMENTS CAN BE ADDED HERE--# BLOCK IMPULSE DATE 01-JAN-2010 6 DEHDUESO203028DJE \ SEQUENCE=ai=0:at=221:ae=3:lu=100:lo=NNU:ei=1021055:lr=1: \ USERID=ID=291821 NO_USERS=3 GROUP=ONE id_info=1021055 \ CREATION_DATE=27-JUNE-2013 SN=1021055 KEY ="22WS \ DE34 43RE ED54 GT65 HY67 AQ12 ES23 54CD 87BG 98VC \ 4325 BG56" BLOCK PASSION DATE 01-JAN-2010 6 DEHDUESO203028DJE \ SEQUENCE=ai=0:at=221:ae=3:lu=100:lo=NNU:ei=324356:lr=1: \ USERID=ID=291821 NO_USERS=1 GROUP=ONE id_info=324356 \ CREATION_DATE=27-MAY-2012 SN=324356 KEY ="22WS \ DE34 43RE 342E WSEW T54R HY67 TFRT 4ER4 WE23 XS21 \ CD32 12QW" BLOCK VICTOR DATE 01-JAN-2010 6 DEHDUESO203028DJE \ SEQUENCE=ai=0:at=221:ae=3:lu=100:lo=NNU:ei=324356:lr=1: \ USERID=ID=291821 NO_USERS=5 GROUP=ONE id_info=324356 \ CREATION_DATE=27-MAY-2012 SN=324356 KEY ="22WS \ DE34 43RE 342E WSEW T54R HY67 TFRT 4ER4 WE23 XS21 \ CD32 12QW" #--BLOCK--ENDS--HERE# #--NEW--BLOCKS--CAN--BE--APPENDED--HERE--# 

I am only interested in Block Name , NO_USERS, and id_info of each block . these three data to be saved to a data-structure(lets say dict), which is further stored in a list :

any other data structure which can hold the info would also be fine.

So far i have tried getting the block names by reading line by line :

fOpen = open('DATA.txt') unique =[] for row in fOpen: if "BLOCK" in row: unique.append(row.split()[1]) print unique 

i am thinking of regular expression approach, but i have no idea where to start with. Any help would be appreciate.Meanwhile i am also trying , will update if i get something . Please help .

You could use groupy to find each block, use a regex to extract the info and put the values in dicts:

from itertools import groupby import re with open("test.txt") as f: data = [] # find NO_USERS= 1+ digits or id_info= 1_ digits r = re.compile("NO_USERS=\d+|id_info=\d+") grps = groupby(f,key=lambda ****.****().startswith("BLOCK")) for k,v in grps: # if k is True we have a block line if k: # get name after BLOCK name = next(v).split(None,2)[1] # get lines after BLOCK and get the second of those t = next(grps)[1] # we want two lines after BLOCK _, l = next(t), next(t) d = dict(s.split("=") for s in r.findall(l)) # add name to dict d["Name"] = name # add sict to data list data.append(d) print(data) 

Or without groupby as your file follows a format we just need to extract the second line after the BLOCK line:

with open("test.txt") as f: data = [] r = re.compile("NO_USERS=\d+|id_info=\d+") for line in f: # if True we have a new block if line.startswith("BLOCK"): # call next twice to get thw second line after BLOCK _, l = next(f), next(f) # get name after BLOCK name = line.split(None,2)[1] # find our substrings from l d = dict(s.split("=") for s in r.findall(l)) d["Name"] = name data.append(d) print(data) 

To extract values you can iterate:

for dct in data: print(dct["NO_USERS"]) 

If you want a dict of dicts and to access each section from 1-n you can store as nested dicts using from 1-n as tke key:

from itertools import count import re with open("test.txt") as f: data, cn = <>, count(1) r = re.compile("NO_USERS=\d+|id_info=\d+") for line in f: if line.startswith("BLOCK"): _, l = next(f), next(f) name = line.split(None,2)[1] d = dict(s.split("=") for s in r.findall(l)) d["Name"] = name data[next(cn)] = d data["num_blocks"] = next(cn) - 1 

from pprint import pprint as pp pp(data) , 2: , 3: , 'num_blocks': 3> 

'num_blocks' will tell you exactly how many blocks you extracted.

How to open a file python Code Example, Get code examples like "how to open a file python" instantly right from your google search results with the Grepper Chrome Extension.

With file open in python

open file in python

import os os.system('filename.extension')

Python Read File – How to Open, Read, and Write to, fhand = open ('daffodils.txt') print (fhand) In the example above, the OS will return the file handle in the variable fhand if open is successful. By default, you can only read the file. Output: The output of a file handle.

Источник