[Akonadi] [Bug 351628] New: Akonadi DAV Resource does not sync all calendars on ownCloud any more

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

Bug ID: 351628
Summary: Akonadi DAV Resource does not sync all calendars on
ownCloud any more
Product: Akonadi
Version: 1.13.0
Platform: Mageia RPMs
OS: Linux
Status: UNCONFIRMED
Severity: critical
Priority: NOR
Component: DAV Resource
Assignee: kdepim-***@kde.org
Reporter: ***@suklje.name

What happens is that every now and again, one or several of the calendars
(and/or contacts) resources on ownCloud stop syncing with Akonadi. Since the
sync works fine with my Jolla phone, I suspect the issue is on KDE’s side.

How this looks in practice is that suddenly and without warning items from some
calendars are missing in KOrganizer, Zanshin, EventList and even Akonadi
Console.

What I already tried to do, that before worked, but since the latest update
doesn’t:
• in Akonadi Console: toggle offline; abort activity; restart resource; toggle
online
• in Akonadi Console: restart Akonadi server
• in Akonadi Console: clone the WebDAV resource and remove the old one
• in Akonadi Console: just remove the old one and create a new one manually
• restarting KDE/X
• restarting the computer (yes, I’m getting desperate!)
• created a fresh new user on my laptop with a clean new home and added my
ownCloud as a new CalDAV resource
• export the iCal file from ownCloud and reimport it to ownCloud, so I have a
duplicate (funnily enough if I open that iCal file in KOrganizer it works)

…all to no avail. The events don’t show in the Akonadi Console.

The current state that I can see in Akonadi Console ↦ Browser is that:
• Contacts shows 673 entries (seems correct)
• “Dogodki” calendar shows 17 entries (seems correct)
• “Faks” calendar shows no entries and 0 size (**FALSE**)
• “FSFE” calendar shows 579 entries (seems correct)
• “Osebno” calendar shows no entries and 0 size (**FALSE!**)
• Rojstni dan/Birthdays show 14 entries

The only thing that I can think of as special for the two non-syncing calendars
is that I renamed them at some point (Študij ⇒ Faks; Opravki ⇒ Osebno) and in
the backend they are still called the old name. But this was long long before
this bug happened and it works fine with the Jolla Sailfish calendar app.

My setup: Mageia 5 (stable); x86_64
Akonadi 1.13.0-4
KDE: 4.14.5
KOrganizer: 4.14.5-1
ownCloud: 8.0.4

I suspect this could be related to: https://bugs.kde.org/show_bug.cgi?id=328734

Confirmed at Akademy, when dvratil (will add to CC) looked at my laptop and
tried to figure out what’s the cause.

Reproducible: Always

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #1 from Matija Šuklje <***@suklje.name> ---
Created attachment 94177
--> https://bugs.kde.org/attachment.cgi?id=94177&action=edit
Akonadi Console ⇒ Debugger tab ⇒ Dav Groupware tab

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #2 from Matija Šuklje <***@suklje.name> ---
Created attachment 94178
--> https://bugs.kde.org/attachment.cgi?id=94178&action=edit
Akonadi Console ⇒ Job Tracker tab

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #3 from Matija Šuklje <***@suklje.name> ---
Created attachment 94179
--> https://bugs.kde.org/attachment.cgi?id=94179&action=edit
console output of `akonadictl restart` (only CalDav related)

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #4 from Matija Šuklje <***@suklje.name> ---
Currently I’m trying this workaround:

1) in ownCloud export calendar to .ics
2) add .ics as Local iCal file in Akonadi/KOrganizer
3) create a new (empty) calendar on ownCloud
4) copy all events and todos from said Local iCal to the new calendar (e.g. via
Akonadi Console)
5) remove the old calendar on ownCloud

After a restart, it seems that it works, but it’s too early to tell yet how
long that will last. For starters, now the biggest calendar I have instead of
containing 1016 items, only contains 202 items.

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

Matija Šuklje <***@suklje.name> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |CONFIRMED
Ever confirmed|0 |1

--- Comment #5 from Matija Šuklje <***@suklje.name> ---
The only odd thing that I found in said calendar was that in one of the events
there was a newline in the title. I removed the newline just in case in step 4)
from my previous comment.

Other then that, I don’t see any oddness.

[Grégory Oestreicher]

https://bugs.kde.org/show_bug.cgi?id=351628

Grégory Oestreicher <***@kamago.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@kamago.net

--- Comment #6 from Grégory Oestreicher <***@kamago.net> ---
Could it be that the calendars that are empty all contain events that are more
than 3 months old?

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #7 from Matija Šuklje <***@suklje.name> ---
(In reply to Grégory Oestreicher from comment #6)

Post by Grégory Oestreicher
Could it be that the calendars that are empty all contain events that are
more than 3 months old?

No.

[Grégory Oestreicher]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #8 from Grégory Oestreicher <***@kamago.net> ---
The logs only show that when the resource tries to synchronize
https://thatfunkyplace.wheremymonkeyis.at/remote.php/caldav/calendars/hook/Študij/
and
https://thatfunkyplace.wheremymonkeyis.at/remote.php/caldav/calendars/hook/opravila/
it doesn't find anything.

Do you have something in the ownCloud logs or the Apache logs? Any error?

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #9 from Matija Šuklje <***@suklje.name> ---
(In reply to Matija Šuklje from comment #4)

Post by Matija Å uklje
1) in ownCloud export calendar to .ics
2) add .ics as Local iCal file in Akonadi/KOrganizer
3) create a new (empty) calendar on ownCloud
4) copy all events and todos from said Local iCal to the new calendar (e.g.
via Akonadi Console)
5) remove the old calendar on ownCloud
After a restart, it seems that it works, but it’s too early to tell yet how
long that will last. For starters, now the biggest calendar I have instead
of containing 1016 items, only contains 202 items.

This workaround seems to have deleted the items on my ownCloud as well now, so
I only have ~200 items instead of ~1000 in it. I suspect that this part is due
to the 3 month issue.

I haven’t looked at the ownCloud and Nginx logs yet. Will do that later,
hopefully today.

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #10 from Matija Šuklje <***@suklje.name> ---
Created attachment 94333
--> https://bugs.kde.org/attachment.cgi?id=94333&action=edit
ownCloud log from the same time

Sadly, I don’t have the Nginx logs from that far back in time, but at least I
could find the ownCloud logs with the same timestamp.

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

Matija Šuklje <***@suklje.name> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #94333|0 |1
is obsolete| |

--- Comment #11 from Matija Šuklje <***@suklje.name> ---
Created attachment 94336
--> https://bugs.kde.org/attachment.cgi?id=94336&action=edit
ownCloud log from around the same time

[Grégory Oestreicher]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #12 from Grégory Oestreicher <***@kamago.net> ---
Thanks for the logs. I'll have to whip up a script to make them a bit more
readable, because they're… wow… hard to read would be an understatement :)

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #13 from Matija Šuklje <***@suklje.name> ---
Thanks for tackling this :)

I tried to help out by just including 2-3 hours around the time when I recorded
the Akonadi logs as well. It’s actually not many entries – just stupidly long
lines.

[Grégory Oestreicher]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #14 from Grégory Oestreicher <***@kamago.net> ---
Well, I'm a bit stuck here. It seems that ownCloud chokes on a malformed date
time. As this happens only for the 'opravila' calendar I'm wondering if there
isn't a malformed event in it somewhere.

If you have access to the owncloud database with a SQL client can you look if
you have an event whose start time would be equal to '-4713010'? Assuming your
table prefix is the default 'oc_' the following request may find it: "SELECT *
FROM oc_clndr_objects WHERE calendardata LIKE '%-4713010%';".

This doesn't explain why the other calendar doesn't sync, but there's nothing
about this one in the logs :(

[Joerg Mertin]

https://bugs.kde.org/show_bug.cgi?id=351628

Joerg Mertin <***@solsys.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@solsys.org

--- Comment #15 from Joerg Mertin <***@solsys.org> ---
Hi folks,

it does not seem to be linked solely to Kontact/Akonadi on the KDE side.
I have the same effect at home (using latest owncloud 8.1.1).

I'm connected using latest Kontact ruinning KUbuntu 15.04 using KDE Frameworks
5.12.0.
My wife however is using Mac OS-X 10.6.x - and the OS-X Calendar application of
that version.

Thing is - both these systems show the described behavior in this bug-report.

Only on her iPad mini and on my Android devices - do we not see this error.

[Nick via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=351628

Nick <***@gmx.us> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@gmx.us

[Grégory Oestreicher via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #16 from Grégory Oestreicher <***@kamago.net> ---
I know this is a bit old, but do you still have this issue? If so could you try
send me anonymized ownCloud logs to see if this is caused by an invalid item?

[somnium via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=351628

somnium <***@stampernet.de> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@stampernet.de

--- Comment #17 from somnium <***@stampernet.de> ---
I'm not quite sure, if this is the same problem, but my owncloud contacts and
calendars are not shown in kontact etc. anymore. In the akonadi console the
resources are present and "ready", so no obvious errors. But there is no entry
in kontact.

I'm using owncloud 9.0 and php7 on the server, and Arch linux with the KDE
Applications 15.12.3 on the client side. The calenders and contacts are shown
in android and ios, as well as thunderbird.

[somnium via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #18 from somnium <***@stampernet.de> ---
I've just recheck the akonadi console and found, that there is no data in the
caldav-resource (0 kb). So it seems, the data is not synced from the server.

[somnium via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #19 from somnium <***@stampernet.de> ---
Created attachment 97951
--> https://bugs.kde.org/attachment.cgi?id=97951&action=edit
Debug Log for akonadi console

Debug log from Akonadi-Console after the update of the owncloud-resource

[somnium via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #20 from somnium <***@stampernet.de> ---
It seems that the problem has something to do with the new owncloud version
9.0.

The urls changed from: https://ADDRESS/remote.php/caldav/ and
https://ADDRESS/remote.php/carddav/ to https://ADDRESS/remote.php/dav/ (caldav
and carddav). I changed this manually in akonadiconsole and the calendars and
contacts appeared again.

I think, this is an easy fix in kontact/kdepim.

Cheers.

[Randy Barlow via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=351628

Randy Barlow <***@electronsweatshop.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@electronsweatshop.com

[Erik Quaeghebeur]

https://bugs.kde.org/show_bug.cgi?id=351628

Erik Quaeghebeur <***@equaeghe.nospammail.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@equaeghe.nospammail.net
Status|CONFIRMED |NEEDSINFO
Resolution|--- |WAITINGFORINFO

--- Comment #21 from Erik Quaeghebeur <***@equaeghe.nospammail.net> ---
Is this still an issue?

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #22 from Matija Šuklje <***@suklje.name> ---
I’ll keep an eye open. I knew I had some out of sync events recently, but can’t
recall if it was via Nextcloud (14) or Google Calendar.

[Bug Janitor Service]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #23 from Bug Janitor Service <bug-***@kde.org> ---
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #24 from Matija Šuklje <***@suklje.name> ---
This is a bug that does not happen on a regular basis or very often.

I would suggest either prolonging the NEEDSINFO time for let’s say 2 months.

[Erik Quaeghebeur]

https://bugs.kde.org/show_bug.cgi?id=351628

Erik Quaeghebeur <***@equaeghe.nospammail.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDSINFO |RESOLVED
Resolution|WAITINGFORINFO |REMIND

--- Comment #25 from Erik Quaeghebeur <***@equaeghe.nospammail.net> ---
Marking this as RESOLVED REMIND instead. This means that if the bug reappears
in a month, a year, whatever, you can comment ask for it to be reopened. But
please check related open bugs at that time to avoid duplication
<https://bugs.kde.org/buglist.cgi?component=DAV%20Resource&product=Akonadi&resolution=--->.
(You'll have to copy/paste the link, as the last part is not detected as part
of it.).

[Matija Šuklje]

https://bugs.kde.org/show_bug.cgi?id=351628

--- Comment #26 from Matija Šuklje <***@suklje.name> ---
(In reply to Erik Quaeghebeur from comment #25)

Post by Erik Quaeghebeur
Marking this as RESOLVED REMIND instead. This means that if the bug
reappears in a month, a year, whatever, you can comment ask for it to be
reopened.

Perfect! Thanks for understanding :)