• Register
1 vote
69 views

Problem:

Hello everyone,

I am trying to create a webscraper which will take links from CSV file and scrape data.

but whenever I run this code it says the object of nonetype has no len().

Below is my code:

import requests
from bs4 import BeautifulSoup
import time
import os
import csv
 
file = {}
final_data = []
 
def read_file():
    global file
    f = open("Data.csv")
    for line in f.readlines():
        data = line.split(",")
        file[data[0]]= data[1]
 
def writedata(alldata1, filename):
    with open("./"+filename, "w")as csvfile:
        csvfile = csv.writer(csvfile, delimiter=",")
        csvfile.writerow("")
        for i in range(0, len(alldata1)):
            csvfile.writerow(alldata1[i])
 
def parse_data():
    global file
    global final_data
    for data_obj in file.keys():
        link = file[data_obj]
        #print(link)
        data = getbyget(link, {})
        soup = BeautifulSoup(data, "html.parser")
        get_details = soup.find_all(class_="box2-body")[0]
        details = get_details.find_all(id="ctl00_ContentPlaceHolder1_Lblproposedenddt")
        name = get_details.find_all(id="ctl00_ContentPlaceHolder1_lblprojectname")
        date = ""
        names = ""
        sublist = []
        for more in details:
            date = more.text
            sublist.append(date)
        for nme in name:
            names = nme.text
            sublist.append(names)
 
        get_state = soup.find_all(class_="box2-body")[1]
        state = get_state.find_all(id="ctl00_ContentPlaceHolder1_lblState")
        city = get_state.find_all(id="ctl00_ContentPlaceHolder1_lbldistrict")
        cities = ""
        states = ""
        for statename in state:
            states = statename.text
            sublist.append(states)
        for citys in city:
            cities = citys.text
            sublist.append(cities)
             
        get_pmptername = soup.find_all(class_="box2-body")[2]
        promotername = get_pmptername.find_all(id="ctl00_ContentPlaceHolder1_Lblpromotername")
        geta = get_pmptername.find_all("a")
        alinks = ""
        promote = ""
        for promoter in promotername:
            promote = promoter.text
            sublist.append(promote)
        for linka in geta:
            alinks = "http://up-rera.in/"+ linka["href"]
            sublist.append(alinks)
            final_data.append(sublist)
            print(final_data)
 
def getbyget(url, values):
    res = requests.get(url, data=values)
    data = res.text
    return data
 
def main():
    read_file()
    datas = parse_data()
    writedata(datas, "UP_DATA.csv")                                                      
main()

What’s going wrong here?

12 7 7
15,250 points

Please log in or register to answer this question.

1 Answer

1 vote

Solution:

Most of the time, a TypeError involving the None object indicates that a function mistakenly returned None because of a missing return statement. As per your program, there is no return statement in parse_data(), which means that datas has value None.

13 9 6
94,240 points

Related questions

0 votes
1 answer 5 views
5 views
Problem: I'm seeing weird behavior on this code: images = dict(cover=[],second_row=[],additional_rows=[]) for pic in pictures: if len(images['cover']) == 0: images['cover'] = pic.path_thumb_l elif len(images['second_row']) < 3: images['second_row'].append(pic.path_thumb_m) ... == 0: TypeError: object of type 'NoneType' has no len() I can't figure out what's wrong in this. Maybe some scope issue?
asked Jan 10 Mashhoodch 5.7k points
0 votes
1 answer 17 views
17 views
I am creating a cookie clicker game, where there is a surface that displays how many cookies I have. Here is my code for drawing text. def draw_text(self, text, font_name, size, color, x, y, align="nw"): font = pg.font.Font(font_name, size) text_surface = ... does self.draw_text('Cookies: {}'.format(len(self.cookie_count)) give me an error? How come the length of self.cookie_count is not printed?
asked Oct 25, 2020 psandprop 2.4k points
0 votes
1 answer 3 views
3 views
Problem: I use Regex to retrieve certain content from a search box on a webpage with selenium.webDriver. searchbox = driver.find_element_by_class_name("searchbox") searchbox_result = re.match(r"^.*(?=(\())", searchbox).group() The code works as long as the search ... get error: AttributeError: 'NoneType' object has no attribute 'group' How can I make the script handle the "No results" situation?
asked Jan 9 Mashhoodch 5.7k points
0 votes
1 answer 70 views
70 views
Using Python Regex to retrieve certain content from a search box on a webpage with selenium.webDriver searchbox = driver.find_element_by_class_name("searchbox") searchbox_result = re.match(r"^.*(?=(\())", searchbox).group() The code works as long as the search box ... is error: AttributeError: 'NoneType' object has no attribute 'group' How can I make the script handle the "No results" situation?
asked Oct 24, 2020 psandprop 2.4k points
0 votes
1 answer 9 views
9 views
New to programming and am unsure why I am getting this error count=int(input ("How many donuts do you have?")) if count <= 10: print ("number of donuts: " ) +str(count) else: print ("Number of donuts: many")
asked Oct 25, 2020 psandprop 2.4k points
2 votes
1 answer 76 views
76 views
Problem: I am new in python programming so I am always confused about what is going wrong with my code. Recently I write a code: class showInformation(object): def __int__(self, name, age): self.name = name self.age = age def showName(self): ... showInformation("XXX") TypeError: showInformation() takes no arguments typeerror: object() takes no parameters Whats wrong In my code can anyone explain?
asked Mar 24, 2020 LizzyM 6.1k points
0 votes
1 answer 8 views
8 views
Problem: Hi! I am new to the world of programming. I am trying to multiply two numbers that will be entered by the user. I wrote the following program: number1 = input("Enter first number: ") number2 = input("Enter second number: ") print(number1," * ",number2," = ", ... = ",number1*number2) TypeError: can't multiply sequence by non-int of type 'str' I have no idea what I am doing wrong. Any help?
asked Dec 21, 2020 Code Learner 5.5k points