Site
Home / Forum / General Discussion / Fix WWE Scrape

Fix WWE Scrape


Posted: 14 Feb 2020 11:01

jaybird
Posts: 23
Joined: Feb 2017

Hi,

the following are not being pulled down when the file is in the correct folder:


2019: https://www.thesportsdb.com/event/672057

I can only assume due to the , in the file name?

2020: https://www.thesportsdb.com/event/672203

- Worlds collide comes down fine (without art work)

@Zag

Posted: 14 Feb 2020 11:05

jaybird
Posts: 23
Joined: Feb 2017

Ontop of that one 1 of the entries for 2019 AEW come down when I run tool to scrape and update.

Is there a way to get a debug log to see why it is not scraping and putting items in the database so it shows on system?


which I guess could be down to someone asking it to be renamed from All Elite Wrestling to AEW.

When you go to AEW the breadcrumb is AEW, then becomes All Elite Wrestling

https://www.thesportsdb.com/event/601301. >>>> Home / Fighting / All Elite Wrestling / 2019 / AEW presents All Out

https://www.thesportsdb.com/league/4563. >>>> Home / Fighting / AEW


so which is the corrections folder structure for this to be scrapped?

AEW/SeasonXYZ/Item.mp4

or.

All Elite Wrestling/SeasonXYZ/item.mp4

Posted: 14 Feb 2020 12:48

zag
Posts: 1,558
Joined: Oct 2016

Yes I had noticed this as well, I have completely normalised all the fighting event names now to remove punctuation now and in the future, let me know if its any better?

Posted: 14 Feb 2020 12:59

jaybird
Posts: 23
Joined: Feb 2017

thx @Zag

any ideas why WWE Royal Rumble doesn't come down?

which folder should be used for AEW as it just doesn't work anymore?

and is there a file on the raspiPi that is written to that I can check to provide more debug information?

Posted: 14 Feb 2020 13:20

jaybird
Posts: 23
Joined: Feb 2017

Update on WWE

Sports > WWE > Season 2000 >
Sports > WWE > Season 2019 >

Both Rumble from 2020, and TLC from 2019 do not get scraped and do not show in Videos Section

Posted: 14 Feb 2020 13:47

jaybird
Posts: 23
Joined: Feb 2017

:

I deleted the folder and cleaned the database. I then set the folder again and tried to scrape it. Only 2 AEW shows are picked up.

This leads me to believe there is a bug in the API.... or an issue with the entires for the shows. Is there anyway to check this via Postman or another API testing application?



I have added 2 test mp4's. I have gone to play Worlds Collide. To my surprise it plays what should be Royal Rumble. Which suggests there is an issue mapping the shows. Perhaps an issue with pulling from the shows?

Posted: 16 Feb 2020 19:00

jaybird
Posts: 23
Joined: Feb 2017

@Zag

So the image issue was because I had a lover version of the SportsDb scraper from : https://github.com/zag2me/metadata.thesportsdb.com

After updating the from one minor version below this one Images now download.

I built a brand new LibreElec box, clean, created AEW folders, put some test video files in with the names from the pages here. It still only pulls down the two files named above ^^ ignore the other 3 that are in the system. What could be the issue here?

25 May 2019 r01 Double Or Nothing

30 Jun 2019 r02 Fyter Festival.
Could we either resolve the Database issue, or just delete the entries and re-add them so that the SportsDB AddOn works as expected?

Posted: 17 Feb 2020 20:36

jaybird
Posts: 23
Joined: Feb 2017

Any other Admins around that can have a look at this please?

Posted: 17 Feb 2020 20:56

curswine
Posts: 143
Joined: Jan 2017

I had a quick look and it appears whoever added the entry for Hell in a Cell used the same date as Clash of Champions, I've now corrected that, try updating the date in the filename and scrape again.

Unfortunately most entries for WWE from 2018 onwards were not added by myself, like pretty much the rest of the events are, so there could be a few issues.

I think we'll start again with the AEW events as I'm not too happy with the naming of a few of the them,it's not too much to correct though thankfully.

I'll get to work on all this tomorrow.

Posted: 17 Feb 2020 21:00

jaybird
Posts: 23
Joined: Feb 2017

cheers, can you see why Royal rumble 2020 pulls down worlds collide instead of correct data please? and why TLC 2019 doesn't pull either :S

Posted: 17 Feb 2020 21:37

curswine
Posts: 143
Joined: Jan 2017

cheers, can you see why Royal rumble 2020 pulls down worlds collide instead of correct data please? and why TLC 2019 doesn't pull either :S

I need to do some more testing, but I don't think the scraper likes it when the UTC date is (correctly) set to a date other than the exact same as the event. Try the Royal Rumble 2020 now and see if it works.

Posted: 18 Feb 2020 20:35

jaybird
Posts: 23
Joined: Feb 2017

Hey,

I can see that some of the names have changed from:

WWE.2020-01-25.Worlds.Collide.2020.mp4

to

WWE.2020.01.25.Worlds.Collide.2020.mp4


What ever you have done has fixed TLC 2019, RR 2020 and Worlds Collide 2020

But Hell In a Cell 2019 and S/Series 2019 do not come down

Posted: 18 Feb 2020 21:17

curswine
Posts: 143
Joined: Jan 2017

Hey,

I can see that some of the names have changed from:

WWE.2020-01-25.Worlds.Collide.2020.mp4

to

WWE.2020.01.25.Worlds.Collide.2020.mp4


What ever you have done has fixed TLC 2019, RR 2020 and Worlds Collide 2020

But Hell In a Cell 2019 and S/Series 2019 do not come down



The new file format is nothing to do with me, it shouldn't affect files being scraped though.

Try scraping Hell in a Cell 2019 and Survivor Series 2019 again, should work now if my theory is correct.

Also I've totally updated the AEW events so some of the file names may need adjusting for that.

Posted: 18 Feb 2020 22:08

jaybird
Posts: 23
Joined: Feb 2017

Hey

Thx for resolving AEW : Works fine now

Re: Hell in a Cell : Working Fine, Survivor Series 2019 however still doesn't pull down

Posted: 18 Feb 2020 22:35

curswine
Posts: 143
Joined: Jan 2017

Hey

Thx for resolving AEW : Works fine now

Re: Hell in a Cell : Working Fine, Survivor Series 2019 however still doesn't pull down


Try one more time, Survivor Series 2019 should work now.

Posted: 19 Feb 2020 15:24

zag
Posts: 1,558
Joined: Oct 2016


WWE.2020-01-25.Worlds.Collide.2020.mp4

to

WWE.2020.01.25.Worlds.Collide.2020.mp4



Shit! that's a bug I introduced. The filename should not actually affect anything though as the scraper uses other fields to match.

Posted: 19 Feb 2020 19:58

curswine
Posts: 143
Joined: Jan 2017



Shit! that's a bug I introduced. The filename should not actually affect anything though as the scraper uses other fields to match.


Don't really think that's a bug, more just an unexpected change to formatting, but everything in my library is scraping fine with the hyphens rather than periods.

Posted: 23 Feb 2020 18:46

jaybird
Posts: 23
Joined: Feb 2017

Thanks all working now

Can you let me know which fields are the offenders and I will correct theses If It happens in the future.

Cheers


Who is Online?

In total there are 68 users online :: 3 registered, 0 hidden and 65 guests (based on users active over the past 5 minutes) Most users ever online was 424 on Fri Nov 10, 2017 9:02 pm

About Us

Discussion forum for TheSportsDB.com site and related topics

Rules

- Be Polite
- Respect other users
- Always post log files with issues
- Try to be helpful
- No Piracy discussion

Showing 0 to 18 (Total: 18)