Skip to main content

View Post [edit]

Poster: tracey pooh Date: Jul 1, 2015 7:39am
Forum: forums Subject: Re: Library View Count Not Updating

for new and old look, feel:

search and favorites and "my library" pages update continuously.

collection pages (eg: GratefulDead, prelinger, movies, texts) cache for an hour (the main page / default view) but any clicked "SORT BY" links are not cached / updated continuously.

so now, there is no intent to update pages only once a month.

a VERY rare set of certain metadata/stats like "VIEWS" (number of streams/plays/downloads) for an individual item (for example a single book) update nightly; and VIEWS (and graphs of number of items and VIEWS over time) for collections of items update either nightly/monthly (sort of depends -- a bit too complicated to totally dissect here/now unless you'd find that helpful).

Reply [edit]

Poster: anarchivist77 Date: Jul 1, 2015 11:11am
Forum: forums Subject: Re: Library View Count Not Updating

Thanks Tracey,

So with regard to my initial question, what I hear you saying is that the Views for Collections (i.e. My Library, My Favorites, lists of search results, etc.) may either update nightly or monthly. Is that correct?

In the cases that Katahdin87 and I are referring to here and in his thread (linked by R Pal above), the view count on the collection level was updating nightly until a few weeks ago, when they suddenly stopped updating. Views on the item level continued to update nightly as normal.

I don't know that you need to dissect the ins and outs of your metadata necessarily, but I think Katahdin87 and I would both appreciate knowing whether this sudden switch from daily to monthly view count updates at the collection level will be permanent.

Does that make sense?

Reply [edit]

Poster: R Pal Date: Jul 1, 2015 2:32pm
Forum: forums Subject: Re: Library View Count Not Updating

Based on the explanation / not,

I now see that I may sometimes not see what used to be seen if I saw it. Oh!


.ereh ni morf ynnuf os kool yttiK dna pordwonS

Reply [edit]

Poster: tracey pooh Date: Jul 16, 2015 6:02pm
Forum: forums Subject: Re: Library View Count Not Updating

ok, an issue was found and is being fixed tonight.

please see
http://archive.org/post/1042731/library-view-count-not-updating

Reply [edit]

Poster: anarchivist77 Date: Jul 16, 2015 6:07pm
Forum: forums Subject: Re: Library View Count Not Updating

Thanks Tracey!

Reply [edit]

Poster: anarchivist77 Date: Jul 10, 2015 11:00am
Forum: forums Subject: Re: Library View Count Not Updating

Hi Tracey,

I wanted to touch base on this issue again. I'm fairly certain that it's been over a month now with no view count updates on My Library, My Favorites, or on search results. View counts still update nightly on the item level as far as I can tell.

As an example, here's my Favorites page: https://archive.org/details/fav-anarchivist77

While it isn't the biggest deal in the world, it is a bit inconvenient to have to drill down to the item level every time to track downloads. I only have a few things to track, but for someone like Katahdin87 who has many librivox recordings, the process can quickly become quite cumbersome.

Like Katahdin87 reports here and in the other thread in the librivox forum, the view count updates were occurring nightly until around late May or so, when all updating seemed to cease for both of us. I suspect we are not the only ones affected by this.

Is there any way that someone on the tech end of IA could investigate?

Thanks again for all the great work you do with this site!

Reply [edit]

Poster: tracey pooh Date: Jul 10, 2015 11:47am
Forum: forums Subject: Re: Library View Count Not Updating

I believe "My Favorites" do not "roll up" view counts -- so the view counts for the list only relate to the list itself -- not to an ongoing sum of all the items in it views.

search results dont show view counts -- only individual items have view counts in search results.

"My Library" is just right now a search result under the hood for your uploaded items. so again, there isn't any kind of total views count for it (like summing up all the views for your uploaded items).

Those numbers can be calculated, but our system is not doing that now...

Reply [edit]

Poster: anarchivist77 Date: Jul 10, 2015 6:00pm
Forum: forums Subject: Re: Library View Count Not Updating

Hi Tracey,

Until late May or so, My Library, My Favorites, search results, etc. were actually totally up item level download counts in the list views. They still show the tally, if you check the Favorites page I linked, but the counts aren't being updated anymore. This is definitely new behavior.

Previously, there was a lag of about 24 hours between the download count being updated on the item page and the count updating on the various list views (favorites, library, search, etc.). This could be verified by checking the view count on the lists a day after checking the view count on the item page and noting that they were the same.

I promise I'm not trying to be argumentative here, but as Katahdin87 and I have been explaining, the behavior we've experienced using the site does not reflect the explanation given. The list view did update the item-level view count after a short lag of around 24 hours or so. The fact that it is not doing so now is new behavior beginning as of late May or so.

To reiterate my earlier question, could someone on the coding end possibly look into this to verify why the behavior has changed, whether or not it is affecting all or some of your end users, and whether or not this change represents a bug in the new system that the developers might like to be informed about?

Thanks.

Reply [edit]

Poster: tracey pooh Date: Jul 11, 2015 8:37pm
Forum: forums Subject: Re: Library View Count Not Updating

we absolutely never tracked search results or "my uploads" views counts.

i'm 99.99% sure we never tracked "my favorites" views counts (and if we did, it was only for a short spell and found it presently untractable with our current system and resources.

i'm the website lead programmer and have been so at archive.org for over 10 years now, so you get a sense of where the comments are coming from 8-) -- no "telephone" or "lost in translation" issues going on from our end.

rolling up groups of items view counts has proved difficult for us (for various reasons) so we've mostly stuck to tracking "standard" collections of items (eg: /details/prelinger or /details/GratefulDead or even /details/texts etc.)
and even then, we've had a fair amount of issues over the years.

best regards,
-tracey

Reply [edit]

Poster: geekcookiespodcast Date: Jul 15, 2015 2:36am
Forum: forums Subject: Re: Library View Count Not Updating

Hi tracey,
good to know you are an expert about the infrastructure!

I'm uploading our podcast audio files to archive and I'm pretti sure the "view" columns was update every 24h until one month or so (I wrote a scraper to have download history for each element).In the search for the element I uploaded :

https://archive.org/search.php?query=uploader%3A%22geekcookiespodcast%40gmail.com%22

the view count is now stuck, but at the single element level is stil updating. It is the same if I ask for json or xml result.

Here the json output:

https://archive.org/advancedsearch.php?q=uploader%3A%22geekcookiespodcast%40gmail.com%22&fl[]=downloads&fl[]=identifier&fl[]=title&sort[]=addeddate+desc&sort[]=&sort[]=&rows=50&page=1&output=json&callback=callback&save=yes

I was excited to get the json output, no more scraping aorund!

Is there something I'm doing wrong? Is a place where I can get the aggregated views count from all the files I upload?

Thanks and thank you for the terrific work behind the scenes!

Reply [edit]

Poster: tracey pooh Date: Jul 15, 2015 7:04am
Forum: forums Subject: Re: Library View Count Not Updating

OK, so I'm confused just by the part you wrote above here:

"the view count is now stuck, but at the single element level is stil updating."

because there is no aggregated view count that we compute or output/produce for an arbitrary search.



Using a JSON indenter and "colordiff" (to make it easy), I'll verify much of my items change tonight by comparing today's output with tomorrow's output of this:

https://archive.org/advancedsearch.php?q=collection%3ApoohBot&fl%5B%5D=downloads&fl%5B%5D=identifier&sort%5B%5D=identifier+asc&sort%5B%5D=&sort%5B%5D=&rows=500&page=1&output=json

Reply [edit]

Poster: tracey pooh Date: Jul 16, 2015 7:35am
Forum: forums Subject: Re: Library View Count Not Updating

OK!

I think I now understand what you are referring to, and there *IS* a new problem I was able to find just now.

So I think what you've been noticing is that individual items in our search engine (ie: not collections, but single items) have had their download counts updated nightly normally, but that stopped semi-recently.

CORRECT! 8-) (that's good and bad news, right?)

We are in process of migrating our search engine. Our current live (soon to be dropped) search engine *just* ran out of avail space in /tmp for what it needs to do each night to update items.

So I've pushed live a workaround and am manually re-running last night's run now (and then every night the normal run should be fine after that).

Sorry for the confusion trying to sort out the issue -- I appreciate you sticking with us and helping us sort out the issue.
It's embarrassing that our user base found and reported the issue before we did -- and I apologize for that.
(We have robo-checks for full disks running 24x7 -- but in this sort case, it hit 100%, fatal-ed, then disk cleaned up very quickly so we never hit a threshold trigger).

I'd make the script more bulletproof to a fail like this -- but we are replacing the Search Engine w/i the next 15-90 days...

Best regards, and thanks for putting up with our little glitches from time to time!
-Tracey

Reply [edit]

Poster: geekcookiespodcast Date: Jul 16, 2015 10:14am
Forum: forums Subject: Re: Library View Count Not Updating

Hi,
thans!

It's really amazing you find time to solve this tiny issue, I can barely imagine the volume of stuff you have to work on in this website.

I'm really happy I can rely on the json now, I really didn't want to write a cumbersone parser (I'm really bad at it :-)

That is an example of what I was tracking for the first of my upload

http://www.charted.co/?{%22dataUrl%22%3A%22http%3A%2F%2Fgeekcookies.brapi.net%2Fgeekcookies_ep_0.csv%22}

Thank you and keep up the good work!

Reply [edit]

Poster: tracey pooh Date: Aug 7, 2015 7:18pm
Forum: forums Subject: Re: Library View Count Not Updating

ok, all the individual shows should be updated now (finally).

happy weekend!

Reply [edit]

Poster: CaseUploader155 Date: Sep 19, 2015 11:51am
Forum: forums Subject: Re: Library View Count Not Updating

I need your help. My view count for the criminal case files I uploaded has stopped at 300. My friends were showing these case files to some lawyers and then the view count stopped at 300 for days and days without any additional update. Please fix the view count on my case files pdf upload please?

https://archive.org/details/USAvHill

Also those documents came from PACER so they are public record and can legally be uploaded under federal law. Just needed to note that. Also I ask that you not Judge the person named in those case files as he is trying to prove Actual Innocence to his charge. Thee documents in that upload are uploaded for the purpose of putting the truth on the public record.

Please I beg of you to fix the view count as I believe the documents are getting a lot of views due to the nature of this criminal case and the fact that the Defendant is persistent in trying to prove Actual Innocence to the crime he was indicted on. I know the views are higher then 300 but it is stuck at 300.

Reply [edit]

Poster: geekcookiespodcast Date: Jul 17, 2015 4:56am
Forum: forums Subject: Re: Library View Count Not Updating

Hi, just to let you know, it seems the script failed in the last night run: the top element "geekcookies_ep_8" show 161 views in the search : https://archive.org/search.php?query=uploader%3A%22geekcookiespodcast%40gmail.com%22&sort=-publicdate and 173 in the element page itself: https://archive.org/details/geekcookies_ep_8 I'll wait for your Search Engine upgrade, thank you!
This post was modified by geekcookiespodcast on 2015-07-17 11:56:16