Python read file as hex

How to read binary files as hex in Python?

The simple solution that worked immediately, was to change «r» to «rb», so:

f=open('data.geno','r') # don't work f=open('data.geno','rb') # works fine 

The code in this case is actually only two binary bites, so one byte contains four data, binary; 00, 01, 10, 11.

If the file is encoded in hex format, shouldn’t each byte be represented by 2 characters? So

c=[] with open('data.geno','rb') as f: b = f.read(2) while b: c.append(b.decode('hex')) b=f.read(2) 

Just an additional note to these, make sure to add a break into your .read of the file or it will just keep going.

def HexView(): with open(, 'rb') as in_file: while True: hexdata = in_file.read(16).hex() # I like to read 16 bytes in then new line it. if len(hexdata) == 0: # breaks loop once no more binary data is read break print(hexdata.upper()) # I also like it all in caps. 

D-slr8 99

import binascii # Open in binary mode (so you don't read two byte line endings on Windows as one byte) # and use with statement (always do this to avoid leaked file descriptors, unflushed files) with open('data.geno', 'rb') as f: # Slurp the whole file and efficiently convert it to hex all at once hexdata = binascii.hexlify(f.read()) 

This just gets you a str of the hex values, but it does it much faster than what you’re trying to do. If you really want a bunch of length 2 strings of the hex for each byte, you can convert the result easily:

hexlist = map(''.join, zip(hexdata[::2], hexdata[1::2])) 

which will produce the list of len 2 str s corresponding to the hex encoding of each byte. To avoid temporary copies of hexdata , you can use a similar but slightly less intuitive approach that avoids slicing by using the same iterator twice with zip :

hexlist = map(''.join, zip(*[iter(hexdata)]*2)) 

For people on Python 3.5 and higher, bytes objects spawned a .hex() method, so no module is required to convert from raw binary data to ASCII hex. The block of code at the top can be simplified to just:

with open('data.geno', 'rb') as f: hexdata = f.read().hex() 

ShadowRanger 131462

Related Query

• How to read tables in multiple docx files in a same folder by python
• How to read and write files in python for Arabic
• How to create python executable from a .py file that need to read and write .txt files in a separate folder
• How to read a maze from an image and convert it to binary values in Python
• How to read binary data via pipe in python
• Python how to read from and write to different files using multiprocessing
• how to read a text file in python and output variables into a csv
• How to watch and and monitor ftp mounted point for new created files using Python
• How to read the yaml file as dictionary and update value using python
• Python how to send a gtp packet using scapy. I tried converting the existing hex stream of gtp message but not able to create it properly
• How to read specific bytes from a binary MP3 file in Python?
• How to append multiple files into one in Amazon’s s3 using Python and boto3?
• How to get recent files written to directory if 2 files are written program should detect 2 files using python
• How to move files listed in a text file from one folder to another in python
• How to read text from a QlineEdit which belongs to a MainWindow class, and use it into a Qthread class using python and pyqt?
• How to create multiple files with different name in Python
• How to extract specific data from many csv files and place into a new single csv in python
• How to read contents of a LibreOffice writer annotation from a python macro
• How to read in url in python and then print each URL on the website?
• How to read variable exported from shell script , in python
• How to build single .exe file with multiple files in python using cx_freeze
• how to optimise reading multiple files in python
• how to fix python login script using txt files to login and register users
• How I can execute python virtual environment files with php shell execute?
• How to read the first line from a file as key and the next 3 lines as a list of values to a dictionary, python
• how to count black and white pixels in binary image using python
• How does python differentiate between string and binary message — TCP Socket send
• How to read multiple text files line by line and send to excel moving to a new column after every file?
• How to find new/changed/removed files in a folder with python 3.x
• How can I read stderr with python if I’m NOT using subprocess?
• How to split a log file into several csv files with python
• Python 3.5: How to read a db of JSON objects
• Is there a way to read all files excluding a defined list of files in python apache beam?
• How can I use Python to locate two files with identical names and check if they’re equal?
• how to ignore a field data while comparing 2 files in python
• How to update directory to look for new files in Python using subprocess.Popen
• How to put files in a zip archive using Python zipfile
• How do I replace bytes in multiple files in one directory recursively with Python 3.5.1?
• how to read text copied from web to txt file using python
• How can I read spaces in a Python string, store their exact positions, and then reinsert them in a new string at those positions?
• python replace string method in read file as binary
• How can I run a python script on many files to get many output files?
• Read Dot Files in Python
• How to sort CSV files by alphabet and highest number. Python 3
• How could I read a Markdown list into a Python OrderedDict?
• How to read timestamp from excel list in python
• how to upload to folder and replace files using python and Google Drive?
• Python — Read and plot only the latest files
• Compatibility of Binary Files between Python and .NET
• How do I write a setup.py and other installation files in Python that set path?

More Query from same tag

• Tkinter Label, TypeError: cannot concatenate ‘str’ and ‘instance’ objects
• Tkinter Button States
• Extracting divs with regular expressions
• Easiest Way to Poll A Web Server
• AttributeError: ‘OAndX’ object has no attribute ‘run’
• reuse class object across module
• Trying to find a specific character of a string in consecutive intervals of 10 using python
• Jupyter Notebook: Print 2d array, where each and every row and column is in a single line
• How to have two networks together in TensorFlow?
• Python3 & Tkinter, list.sort list of objects made of StringVar
• Use C++-Iterators on Python-List with the Python/C API?
• Drop down arrows are not showing
• Unknown interpolation method array using griddata on PYTHON
• Python Date Time — Create New Column if Past Date change to Current Date MMM-YY
• No such element: Unable to locate element error while clicking on a button using Selenium and Python
• Unable to locate xpath element — Selenium, Python, Chromedriver
• How to calculate shortest path lenght of given node faster?
• cant get InstaPy to start
• Keras model.predict gives inconsistent values
• How to make all cells in heatmap square
• Batch converting Corel Paradox 4.0 Tables to CSV/SQL — via PHP or other scripts
• Mocking selective file writes in python unittest
• timeout error when using external terminal in Manjaro
• How to get the name of all columns using openpyxl?
• How to click this button using Selenium through Python
• Scraping website for image path(not downloading the image just getting clickable link) but image url is parsed in scraped text
• Constraint of cvxpy: How to constrain only some elements of the variable to be nonzero?
• Jinja2: using a pandas dataframe or string variable
• how to find a set amount of letters in between symbols
• Print a specific time giving by the user python
• I have a problem while installing Odoo v13. requirements.txt. Related with Microsoft Visual C++ Build Tools
• Error when disconnecting from server
• How to authenticate in Jenkins while remotely accessing its JSON API?
• For loop is running too quickly in my discord.py bot
• WxPthon Question: Subpanels within a Notebook Panel

Источник

Python-сообщество

• Начало
• » Python для новичков
• » Открыть файл в hex

#1 Дек. 20, 2011 19:26:49

Открыть файл в hex

Подскажите как открыть файл (.avi) в hex коде
Надо открыть файл видео в виде кода и парсить на нахождение одинаковых сочетаний кода..

#2 Дек. 20, 2011 20:43:33

Открыть файл в hex

#3 Дек. 20, 2011 21:33:26

Открыть файл в hex

bytelist = []

bytes = open('1.avi','rb')
filedata = bytes.read()

for byte in filedata: 
bytelist.append(byte)
print bytelist

#4 Дек. 20, 2011 21:50:37

Открыть файл в hex

.read — не самый лучший выбор для чтения больших файлов.
И я бы посоветовал использовать bytearray вместо обычного списка:

bytelist = bytearray()
with open('1.avi','rb') as bytes: 
for byte in bytes: 
bytelist.extend(byte)

#5 Дек. 20, 2011 22:10:56

Открыть файл в hex

#6 Дек. 20, 2011 23:12:06

Открыть файл в hex

зачем его вообще в память загружать ?

потому что файл загружается в память

если использовать .read(n), то можно пропустить подходящий фрагмент
например, поиск слова abcd может не сработать, когда
.read(n) -> aaaaaabc
.read(n) -> deeeeeeee
то есть нужный фрагмент есть, но он разделён между строками
(блочное чтение происходит быстрее, чем посимвольное)

используй конечный автомат, который при обнаружении начала переходит в состояние накапливания

Отредактировано (Дек. 20, 2011 23:12:18)

#7 Дек. 21, 2011 15:58:43

Открыть файл в hex

bytelist = []
f = open('1.txt', 'rb').read()
bytelist = [ ord(c) for c in f]
print(bytelist)
file=open('2.dat','wb')
file.write(str(bytelist))
file.close()

все нормально если файл небольшой.
Но при большом объеме тупо виснет, понимаю что идет в память .
Если объясню задачу может лучше будет:
надо в коде найти одинаковые сочетания,не заданные ,например(112), вырезать их поставив
типа маркера на место вырезанного кода,записать эти вырезанные участки например(112)-код,(4)-тип маркера, в списки, в конечном итоге вставить все на место из этих списков и все-на выходе получить тот же самый файл
Все это должно напоминать архиватор в некотором смысле

#8 Дек. 22, 2011 19:36:24

Открыть файл в hex

Извиняюсь работает скрипт

bytelist = bytearray()
with open('01.mp3','rb') as bytes: 
for byte in bytes: 
bytelist.extend(byte) 

file=open('2.dat','wb')
file.write(str(bytelist))
file.close()

#9 Дек. 22, 2011 20:21:37

Открыть файл в hex

скриптом выношу из бинарного во временной файл hex

bytelist = bytearray()
with open('01.mp3','rb') as bytes: 
for byte in bytes: 
bytelist.extend(byte) 

file=open('2.dat','wb')
file.write(str(bytelist))
file.close()


file=open('2.dat','rb')
data=file.read()
file.close()

file=open('2.dat','wb')
file.write(str(data.encode('hex')))
file.close()

подскажите алгоритм поиска одинаковых сочетаний(например пять единиц или семь F), потом надо вырезать это из кода,поставив допустим точку и цифру 1(типа метки),а сочетание вынести в начало временного файла ,(типа (.1)-это пять единиц)

#10 Дек. 22, 2011 20:50:52

Открыть файл в hex

может воспользоваться для записи количества повторений itertools.groupby ?
И кстати может не надо изобретать велосипеды вам просто RLE не подойдет?
http://www.inference.phy.cam.ac.uk/mackay/python/compress/

Отредактировано (Дек. 22, 2011 20:52:47)

Источник